\documentclass{article}
\input{../macros.tex}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Lecture 7:  {\large Solving $\mb A\vb x = \vb 0$: pivot variables, special solutions}}

We have a definition for the column space and the nullspace of a matrix,
but how do we compute these subspaces?

\subsection*{Computing the nullspace}

The {\em nullspace} of a matrix $\mb A$ is made up of the vectors
$\vb x$ for which $\mb A \vb x = \vb 0$.  

Suppose:
$$\mb A = \left[ \begin{array}{rrrr} 1 & 2 & 2 & 2\\
2 & 4 & 6 & 8\\
3 & 6 & 8 & 10 \end{array} \right].$$
(Note that the columns of this matrix $\mb A$ are not independent.)
Our algorithm for computing the nullspace of this matrix uses the
method of elimination, despite the fact that $\mb A$ is not
invertible.  We don't need to use an augmented matrix because the right
side (the vector $\vb b$) is $\vb 0$ in this computation.

The row operations used in the method of elimination don't change the
solution to $\mb A \vb x = \vb b$ so they don't change the nullspace.
(They do affect the column space.)

\smallskip

The first step of elimination gives us:
$$
\mb A = \left[ \begin{array}{rrrr} 1 & 2 & 2 & 2\\
2 & 4 & 6 & 8\\
3 & 6 & 8 & 10 \end{array} \right] \quad \longrightarrow \quad
\left[ \begin{array}{rrrr} 1 & 2 & 2 & 2\\
0 & 0 & 2 & 4\\
0 & 0 & 2 & 4 \end{array} \right].
$$
We don't find a pivot in the second column, so our next pivot is the $2$ in the third column of the second row:
$$
\left[ \begin{array}{rrrr} 1 & 2 & 2 & 2\\
0 & 0 & 2 & 4\\
0 & 0 & 2 & 4 \end{array} \right]
\quad \longrightarrow \quad
\left[ \begin{array}{rrrr} 1 & 2 & 2 & 2\\
0 & 0 & 2 & 4\\
0 & 0 & 0 & 0 \end{array} \right] = \mb U
$$ The matrix $\mb U$ is in {\em
  echelon} (staircase) form.  The third row is zero because row
$3$ was a linear combination of rows $1$ and $2$; it was eliminated.

The {\em rank} of a matrix $\mb A$ equals the number of pivots it has.  In this example, the rank of
$\mb A$ (and of $\mb U$) is $2$.

\subsection*{Special solutions}

Once we've found $\mb U$ we can use back-substitution to find the
solutions $\vb x$ to the equation $\mb U \vb x = \vb 0$.
In our example, columns $1$ and $3$ are {\em pivot columns} containing
pivots, and columns $2$ and $4$ are {\em free columns}.  We can assign
any value to $x_2$ and $x_4$; we call these {\em free variables}.
Suppose $x_2 = 1$ and $x_4 = 0$.  Then:
$$2x_3 + 4x_4 = 0 \quad \Longrightarrow \quad x_3 = 0$$
and:
$$x_1 + 2x_2 + 2x_3 + 2x_4 = 0 \quad \Longrightarrow \quad x_1 = -2.$$
So one solution is $\vb x = \left[ \small \begin{array}{r} -2 \\ 1 \\ 0
    \\ 0 \end{array} \right]$ (because the second column is just twice the first
  column).  Any multiple of this vector is in the
  nullspace.

Letting a different free variable equal $1$ and setting the other free
variables equal to zero gives us other vectors in the nullspace.  For
example: $$\vb x = \left[ \small \begin{array}{r} 2 \\ 0 \\ -2
    \\ 1 \end{array}\right]$$ has $x_4 = 1$ and $x_2 = 0$.  The nullspace of $\mb A$ is the
collection of all linear combinations of these ``special solution''
vectors.

The rank $r$ of $\mb A$ equals the number of pivot columns, so the
number of free columns is $n-r$: the number of columns (variables) minus
the number of pivot columns.  This equals the number of special
solution vectors and the dimension of the nullspace.

\subsection*{Reduced row echelon form}

By continuing to use the method of elimination we can convert $\mb U$
to a matrix $\mb R$ in {\em reduced row echelon form} (rref form),
with pivots equal to $1$ and zeros above and below the pivots.
$$
\mb U = \left[\begin{array}{rrrr} 1 & 2 & 2 & 2\\ 0 & 0 & 2 & 4 \\
0 & 0 & 0 & 0\end{array}\right] \; \rightarrow \;
\left[\begin{array}{rrrr} 1 & 2 & 0 & -2\\ 0 & 0 & 2 & 4 \\
0 & 0 & 0 & 0\end{array}\right]  \; \rightarrow \;
\left[\begin{array}{rrrr} 1 & 2 & 0 & -2\\ 0 & 0 & 1 & 2 \\
0 & 0 & 0 & 0\end{array}\right] = \mb R.
$$ 

By exchanging some columns, $\mb R$ can be rewritten with a copy of the
identity matrix in the upper left corner, possibly followed by some
free columns on the right.
 If some rows of $\mb A$ are linearly
dependent, the lower rows of the matrix $\mb R$ will be filled with
zeros:
$$\mb R = \left[\begin{array}{rr} \mb I & \mb F \\ \mb 0 & \mb
    0\end{array}\right].$$
(Here $\mb I$ is an $r$ by $r$ square matrix.)
 
If $\mb N$ is the {\em nullspace matrix} $\mb N = \twobyone {- \mb F}
{\mb I}$ then $\mb R \mb N = \mb 0$.  (Here $\mb I$ is an
$n-r$ by $n-r$ square matrix and $\mb 0$ is an $m$ by $n-r$ matrix.)
The columns of $\mb N$ are the special solutions.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}