\documentclass{article}
\input{../macros.tex}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Lecture 6:  Column space and nullspace}

In this lecture we continue to study subspaces, particularly the
column space and nullspace of a matrix.

\subsection*{Review of subspaces}

A vector space is a collection of vectors which is closed under linear
combinations.  In other words, for any two vectors
$\vb v$ and $\vb w$ in the space and any two real numbers $c$ and $d$,
the vector $c\vb v + d\vb w$ is also in the vector space.  A subspace
is a vector space contained inside a vector space.

A plane $P$ containing $\small \threebyone 000$ and a line $L$ containing $\small \threebyone 000$ are
both subspaces of $\R^3$.  The union $P \cup L$ of those two subspaces
is generally not a subspace, because the sum of a vector in $P$ and a
vector in $L$ is probably not contained in $P \cup L$.  The
intersection $S \cap T$ of two subspaces $S$ and $T$ is a subspace.
To prove this, use the fact that both $S$ and $T$ are closed under
linear combinations to show that their intersection is closed under
linear combinations.

\subsection*{Column space of $\mb A$}

The {\em column space} of a matrix $\mb A$ is the vector space made up
of all linear combinations of the columns of $\mb A$.

\subsubsection*{Solving $\mb A \vb x = \vb b$}

Given a matrix $\mb A$, for what vectors $\vb b$ does $\mb A \vb x =
\vb b$ have a solution $\vb x$?

$$\mbox{Let }\mb A = \left[ \begin{array}{rrr} 1& 1 & 2\\ 2 & 1 &
    3\\ 3 & 1 & 4\\ 4 & 1 & 5\end{array}\right].$$ Then $\mb A \vb x =
\vb b$ does not have a solution for every choice of $\vb b$ because
solving $\mb A \vb x = \vb b$ is equivalent to solving four linear
equations in three unknowns.  If there is a solution $\vb x$ to $\mb A
\vb x = \vb b$, then $\vb b$ must be a linear combination of the
columns of $\mb A$.  Only three columns cannot fill the entire four
dimensional vector space -- some vectors $\vb b$ cannot be expressed
as linear combinations of columns of $\mb A$.

Big question: what $\vb b$'s allow $\mb A \vb x = \vb b$ to be solved?

A useful approach is to choose $\vb x$ and find the vector
$\vb b = \mb A \vb x$ corresponding to that solution.  The components of  $\vb x$ are just the coefficients in a linear
combination of columns of $\mb A$.

The system of linear equations $\mb A \vb x = \vb b$ is {\em solvable}
exactly when $\vb b$ is a vector in the {\em column space} of $\mb A$.

\smallskip

For our example matrix $\mb A$, what can we say about the column space of $\mb A$?  Are the columns of
$\mb A$ {\em independent}?  In other words, does each column contribute something new
to the subspace?

The third column of $\mb A$ is the sum of the first two columns, so does not add
anything to the subspace.  The column space of our matrix $\mb A$ is a two dimensional subspace of $\R^4$.

\subsection*{Nullspace of $\mb A$}

The {\em nullspace} of a matrix $\mb A$ is the collection of all solutions
$\vb x = \small \threebyone {x_1} {x_2} {x_3}$ to the equation $\mb A
\vb x = \mb 0$.

The column space of the matrix in our example was a subspace of $\R^4$.  The nullspace
of $\mb A$ is a subspace of $\R^3$.  To see that it's a vector space, check that any sum
or multiple of solutions to $\mb A \vb x = \vb 0$ is also a solution: $\mb A(\vb x_1 + \vb x_2) = \mb A \vb x_1 + \vb x_2 = \vb 0 + \vb 0$ and $\mb A (c \vb x) = c \mb A \vb x = c(\vb 0)$.

In the example:
$$ \left[ \begin{array}{rrr} 1& 1 & 2\\ 2 & 1 & 3\\ 3 & 1 & 4\\ 4 & 1
    & 5\end{array}\right] \threebyone {x_1} {x_2} {x_3} =
\fourbyone 0 0 0 0,$$ the nullspace $N(\mb A)$ consists of all multiples of $\small \threebyone 1 1
{-1}$; column 1 plus column 2 minus column 3 equals the zero vector.  This nullspace is a line in $\R^3$.

\subsubsection*{Other values of $\vb b$}

The solutions to the equation:
$$ \left[ \begin{array}{rrr} 1& 1 & 2\\ 2 & 1 & 3\\ 3 & 1 & 4\\ 4 & 1
    & 5\end{array}\right] \threebyone {x_1} {x_2} {x_3} =
\fourbyone 1 2 3 4$$ do not
form a subspace.  The zero vector is not a solution to this equation.  The set of
solutions forms a line in $\R^3$ that passes through the
points $\small \threebyone 1 0 0$ and $\small \threebyone 0 {-1} 1$ but not $\small \threebyone 000$.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}