\documentclass{article}
\input{../macros.tex}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Lecture 9:  Independence, basis, and dimension}

What does it mean for vectors to be independent?  How does the
idea of independence help us describe subspaces like the nullspace?

\subsection*{Linear independence}

Suppose $\mb A$ is an $m$ by $n$ matrix with $m < n$ (so $\mb A \vb x =
\vb b$ has more unknowns than equations).  $\mb A$ has at least one free
variable, so there are nonzero solutions to $\mb A \vb x = \vb 0$.  A combination of the columns is zero, so the columns of this $\mb A$ are {\em dependent}.

We say vectors $\vb x_1, \vb x_2, ... \vb x_n$ are {\em linearly independent} (or just {\em independent})
if $c_1 \vb x_1 + c_2\vb x_2+ \cdots +
c_n \vb x_n = \vb 0$ only when $c_1, c_2, ..., c_n$ are all $0$.
When those vectors are the columns of $\mb A$, the only solution to $\mb A\vb x = \vb 0$ is $\vb x = \vb 0$.

Two vectors are independent if they do not lie on the same line.
Three vectors are independent if they do not lie in the same plane.
Thinking of $\mb A \vb x$ as a linear combination of the column
vectors of $\mb A$, we see that the column vectors of $\mb A$ are
independent exactly when the nullspace of $\mb A$ contains only the
zero vector.

If the columns of $\mb A$ are independent then all columns are pivot
columns, the rank of $\mb A$ is $n$, and there are no free
variables.  If the columns of $\mb A$ are dependent then the rank of
$\mb A$ is less than $n$ and there are free variables.

\subsection*{Spanning a space}

Vectors $\vb v_1, \vb v_2, ... \vb v_k$ {\em span} a space when the space
consists of all combinations of those vectors.  For example, the
column vectors of $\mb A$  span the column space of $\mb A$.

If vectors $\vb v_1, \vb v_2, ... \vb v_k$ span a space $S$, then $S$ is the
smallest space containing those vectors.

\subsection*{Basis and dimension}

A {\em basis} for a vector space is a sequence of vectors $\vb v_1,
\vb v_2, ... \vb v_d$ with two properties:
\begin{itemize}
\item $\vb v_1, \vb v_2, ... \vb v_d$ are independent
\item $\vb v_1, \vb v_2, ... \vb v_d$ span the vector space.
\end{itemize}
The basis of a space tells us everything we need to know about that space.

\subsubsection*{Example:  $\R^3$}

One basis for $\R^3$ is $\left\{ \small \threebyone 100,  \small \threebyone 010,
  \small \threebyone 001 \right\}$.  These are independent because: $$c_1
 \small \threebyone 100 + c_2  \small \threebyone 010 + c_3  \small \threebyone 001 =
 \small \threebyone 000$$ is only possible when $c_1 = c_2 = c_3 = 0$.  These vectors span $\R^3.$

As discussed at the start of Lecture 10, the vectors $\small
\threebyone 112, \small \threebyone 225$ and $\small \threebyone 338$
do not form a basis for $\R^3$ because these are the column vectors of a matrix
that has two identical rows.  The three vectors are not linearly
independent.

In general, $n$ vectors in $\R^n$ form a basis if they are the column
vectors of an invertible matrix.

\subsubsection*{Basis for a subspace}

The vectors $\small \threebyone 112$ and $\small \threebyone 225$ span a plane in
$\R^3$ but they cannot form a basis for $\R^3$.  Given a space, every
basis for that space has the same number of vectors; that number is
the {\em dimension} of the space.  So there are exactly $n$ vectors in
every basis for $\R^n$.

\subsection*{Bases of a column space and nullspace}

Suppose:
$$\mb A = \left[ \begin{array}{rrrr} 1 & 2 & 3 & 1\\ 1 & 1 & 2 & 1\\ 1& 2 &
    3 & 1\end{array}\right].$$ By definition, the four column vectors
of $\mb A$ span the column space of $\mb A$.  The third and fourth
column vectors are dependent on the first and second, and the first
two columns are independent.  Therefore, the first two column vectors
are the pivot columns.  They form a basis for the column space $C(\mb
A)$.  The matrix has rank $2$.  In fact, for any matrix $\mb A$ we can say:
$$\mbox{rank}(\mb A) = \mbox{number of pivot columns of }\mb A = \mbox{dimension of } C(\mb A).$$ (Note that matrices have a rank but not
a dimension.  Subspaces have a dimension but not a rank.)

The column vectors of this $\mb A$ are not independent, so the nullspace $N(\mb A)$
contains more than just the zero vector.  Because the third column is
the sum of the first two, we know that the vector $\small \fourbyone {-1} {-1} 1 0$ is in
the nullspace.  Similarly, $\small \fourbyone {-1} 0 0 {-1}$ is also in $N(\mb A)$.  These are
the two special solutions to $\mb A \vb x = \vb 0$.
We'll see that:
$$\mbox{dimension of } N(\mb A) = \mbox{number of free variables} = n-r,$$ so we
know that the dimension of $N(\mb A)$ is $4-2=2$.  These two special
solutions form a basis for the nullspace.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}