\documentclass{article}
\input{../macros.tex}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Lecture 10: The four fundamental subspaces}

In this lecture we discuss the four fundamental spaces associated with
a matrix and the relations between them.

\subsection*{Four subspaces}

Any $m$ by $n$ matrix $\mb A$ determines four subspaces (possibly containing only the zero vector):

\subsubsection*{Column space, $C(\mb A)$}

$C(\mb A)$ consists of all combinations of the columns of $\mb A$ and
  is a vector space in $\R^m$.

\subsubsection*{Nullspace, $N(\mb A)$}

This consists of all solutions $\vb x$ of the equation $\mb A \vb x =
\vb 0$ and lies in $\R^n$.

\subsubsection*{Row space, $C(\mb A^T)$}

The combinations of the row vectors of $\mb A$ form a subspace of $\mb
R^n$.  We equate this with $C(\mb A^T)$, the column space of the transpose of $\mb A$.

\subsubsection*{Left nullspace, $N(\mb A^T)$}

We call the nullspace of $\mb A^T$ the {\em left nullspace} of $\mb
A$.  This is a subspace of $\R^m$.



\subsection*{Basis and Dimension}

\subsubsection*{Column space}

The $r$ pivot columns form a basis for $C(\mb A)$
$$\mbox{dim } C(\mb A) = r.$$

\subsubsection*{Nullspace}

The special solutions to $\mb A \vb x = \vb 0$ correspond to free
variables and form a basis for $N(\mb A)$.  An $m$ by $n$ matrix has
$n-r$ free variables:
$$\mbox{dim } N(\mb A) = n-r.$$

\subsubsection*{Row space}

We could perform row
reduction on $\mb A^T$, but instead we make use of $\mb R$, the row
reduced echelon form of $\mb A$.  
$$\mb A = \left[ \begin{array}{rrrr} 1 & 2 & 3 & 1\\ 1 & 1 & 2 & 1\\ 1& 2 &
    3 & 1\end{array}\right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrrr} 1 & 0 & 1 & 1\\ 0 & 1 & 1 & 0\\ 0 & 0 &
    0 & 0\end{array}\right] = \left[ \begin{array}{rr} \mb I & \mb F \\ \mb 0 & \mb 0 \end{array} \right] = \mb R$$
Although the column spaces of $\mb A$ and $\mb R$  are
different, the row space of $\mb R$ is the same as the
row space of $\mb A$.  The rows of $\mb R$ are combinations of the
rows of $\mb A$, and because reduction is reversible the rows of $\mb A$ are combinations of the rows of $\mb R$.

The first $r$ rows of $\mb R$ are the "echelon" basis for the row space
of $\mb A$:
$$\mbox{dim } C(\mb A^T) = r.$$

\subsubsection*{Left nullspace}

The matrix $\mb A^T$ has $m$ columns.  We just saw that $r$ is the rank of $\mb A^T$, so  the number of free columns
of $\mb A^T$ must be  $m-r$:
$$\mbox{dim } N(\mb A^T) = m-r.$$

The left nullspace is the collection of vectors $\mb y$ for which $\mb
A^T \mb y = \mb 0$.  Equivalently, $\mb y^T \mb A = \mb 0$; here $\mb
y$ and $\mb 0$ are row vectors.  We say ``left nullspace'' because
$\mb y^T$ is on the left of $\mb A$ in this equation.

To find a basis for the left nullspace we reduce an augmented version
of $\mb A$:
$$\onebytwo {\mb A_{m \times n}} {\mb I_{m \times n}}
\longrightarrow \onebytwo {\mb R_{m \times n}}  {\mb E_{m \times n}}.$$  From this we get the matrix $\mb E$ for which $\mb E \mb A = \mb
R$.  (If $\mb A$ is a square, invertible matrix then $\mb E = \mb
A^{-1}$.)  In our example,
$$\mb E \mb A = \threebythree {-1} 2 0 1 {-1} 0 {-1} 0 1
\left[ \begin{array}{rrrr} 1 & 2 & 3 & 1\\ 1 & 1 & 2 & 1\\ 1& 2 & 3 &
    1\end{array}\right] = \left[ \begin{array}{rrrr} 1 & 0 & 1 & 1\\ 0
    & 1 & 1 & 0\\ 0 & 0 & 0 & 0\end{array}\right] = \mb R.$$ The
bottom $m-r$ rows of $\mb E$ describe linear dependencies of rows of
$\mb A$, because the bottom $m-r$ rows of $\mb R$ are zero.  Here $m-r = 1$ (one zero row in $\mb R$).

The bottom $m-r$ rows of $\mb E$ satisfy the equation $\vb y^T \mb A = \vb 0$ and form a basis for the left nullspace
of $\mb A$.

\subsection*{New vector space}

The collection of all $3 \times 3$ matrices forms a vector space; call
it $M$.  We can add matrices and multiply them by scalars and there's a
zero matrix (additive identity).  If we
ignore the fact that we can multiply matrices by each other, they
behave just like vectors.

Some subspaces of $M$ include:
\begin{itemize}
\item all upper triangular matrices
\item all symmetric matrices
\item $D$, all diagonal matrices 
\end{itemize}
$D$ is the intersection of the first two spaces.  Its dimension is $3$; one basis for $D$ is:
$$\threebythree 100000000, \threebythree 100030000, \threebythree 000000007.$$

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}