\documentclass{article}
\input{../macros.tex}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Lecture 8: Solving $\mb A\vb x = \vb b$: row reduced form $\mb R$}

When does
$\mb A \vb x = \vb b$ have solutions $\vb x$, and how can we describe those solutions?

\subsection*{Solvability conditions on $\vb b$}

We again use the example:
$$\mb A = \left[ \begin{array}{rrrr} 1 & 2 & 2 & 2\\ 2 & 4 & 6 & 8\\ 3
    & 6 & 8 & 10 \end{array} \right].$$ The third row of $\mb
A$ is the sum of its first and second rows, so we know that if $\mb A \vb
x = \vb b$ the third component of $\vb b$ equals the sum of its
first and second components.  If $\vb b$ does not satisfy $b_3 = b_1 +
b_2$ the system has no solution.  If a combination of the rows of $\mb
A$ gives the zero row, then the same combination of the entries of
$\vb b$ must equal zero.

One way to find out whether $\mb A \vb x = \vb b$ is solvable is to use
elimination on the augmented matrix.  If a row of
$\mb A$ is completely eliminated, so is the corresponding entry in
$\vb b$.  In our example, row 3 of $\mb A$ is completely eliminated:
$$\left[ \begin{array}{rrrrr} 
1 & 2 & 2 & 2 & b_1\\ 
2 & 4 & 6 & 8 & b_2\\ 
3 & 6 & 8 & 10 & b_3 \end{array} \right] 
\; \rightarrow \; \cdots \; \rightarrow \; 
\left[ \begin{array}{rrrrl} 
1 & 2 & 2 & 2 & b_1\\ 
0 & 0 & 2 & 4 & b_2 - 2b_1\\ 
0 & 0 & 0 & 0 & b_3 - b_2 - b_1
\end{array} \right].$$
If $\mb A \vb x = \vb b$ has a solution, then 
$b_3 - b_2 - b_1 = 0$.  For example, we could choose $\vb b = \small
\threebyone 1 5 6$.

From an earlier lecture, we know that $\mb A \vb x = \vb b$ is
solvable exactly when $\vb b$ is in the column space $C(\mb A)$.  We have these two conditions on $\vb b$; in fact they are equivalent.

\subsection*{Complete solution}

In order to find all solutions to $\mb A \vb x = \vb b$ we first check
that the equation is solvable, then find a particular solution.  We
get the complete solution of the equation by adding the particular
solution to all the vectors in the nullspace.

\subsubsection*{A particular solution}

One way to find a particular solution to the equation $\mb A \vb x =
\vb b$ is to set all free variables to zero, then solve for the pivot
variables.

For our example matrix $\mb A$, we let $x_2 = x_4 = 0$ to get the system of equations:
\begin{eqnarray*}
x_1 + 2x_3 &=& 1\\
2x_3 &=& 3
\end{eqnarray*}
which has the solution $x_3 = 3/2$, $x_1 = -2$.  Our particular solution is:
$$\vb x_p = \fourbyone {-2} 0 {3/2} 0.$$

\subsubsection*{Combined with the nullspace}

The general solution to $\mb A \vb x = \vb b$ is given by $\vb
x_{\mbox{\footnotesize complete}} = \vb x_p + \vb x_n$, where $\vb x_n$ is a generic
vector in the nullspace.  To see this, we add $\mb A \vb x_p = \vb
b$ to $\mb A \vb x_n = \vb 0$ and get $\mb A \left(\vb x_p +
\vb x_n\right) = \vb b$ for every vector $\vb x_n$ in the nullspace.

Last lecture we learned that the nullspace of $\mb A$ is the
collection of all combinations of the special solutions $\small \fourbyone {-2} 1 0 0$ and
$\small \fourbyone 2 0 {-2} 1$.  So the complete solution to the equation
$\mb A \vb x = \small \threebyone 1 5 6$ is:
$$\vb x_{\mbox{\footnotesize complete}} = 
\fourbyone {-2} 0 {3/2} 0 +  
\fourbyone {-2} 1 0 0 + 
c_2 \fourbyone 2 0 {-2} 1,$$
where $c_1$ and $c_2$ are real numbers.

The nullspace of $\mb A$ is a two dimensional subspace of $\R^4$, and
the solutions to the equation $\mb A \vb x = \vb b$ form a plane
parallel to that through $x_p = \small \fourbyone {-2} 0 {3/2} 0$.

\subsection*{Rank}

The rank of a matrix equals the number of pivots of that matrix.  If $\mb A$ is an $m$ by $n$ matrix of rank $r$, we know $r \leq m$ and
$r \leq n$.

\subsubsection*{Full column rank}

If $r = n$, then from the previous lecture we know that the nullspace
has dimension $n-r = 0$ and contains only the zero vector.  There are
no free variables or special solutions.  

If $\mb A \vb x = \vb b$ has a solution, it is unique; there is either
$0$ or $1$ solution.  Examples like this, in which the columns are
independent, are common in applications.

We know $r \leq m$, so if $r=n$ the number of columns of the matrix is less than
or equal to the number of rows.  The row reduced echelon form of the
matrix will look like $\mb R = \twobyone {\mb I} {\mb 0}$.  For any
vector $\vb b$ in $\R^m$ that's not a linear combination of the
columns of $\mb A$, there is no solution to $\mb A \vb x = \vb
b$.

\subsubsection*{Full row rank}

If $r = m$, then the reduced matrix $\mb R = \onebytwo {\mb I} {\mb F}$ 
has no rows of zeros and so there are no requirements for
the entries of $\vb b$ to satisfy.  The equation $\mb A \vb x = \vb b$
is solvable for every $\vb b$.  There are $n-r = n-m$ free variables,
so there are $n-m$ special solutions to $\mb A \vb x = \vb 0$.

\subsubsection*{Full row and column rank}

If $r = m = n$ is the number of pivots of $\mb A$, then $\mb A$ is an
invertible square matrix and $\mb R$ is the identity matrix.  The nullspace has dimension zero, and $\mb A \vb x = \vb b$ has a unique solution for every $\vb b$ in $\R^m$.

\subsubsection*{Summary}

If $\mb R$ is in row reduced form with pivot columns first (rref), the table below summarizes our results.

\smallskip

\noindent
\begin{tabular}{|p{.59in} | c|c|c|c|}
\hline
 & $r = m = n$ & $r = n < m$ & $r = m < n$ & $r < m, r < n$\\
\hline
& & & &\\
$\mb R$ &  $\mb I$ & 
$\twobyone {\mb I} {\mb 0}$ &
$ \onebytwo {\mb I} {\mb F}$ &
$\twobytwo {\mb I} {\mb F} {\mb 0} {\mb 0}$\\
& & & &\\
{\small \# solutions to $\mb A \vb x = \vb b$} & $1$ & $0$ {\small or} $1$ &  {\small infinitely many} &  $0$ {\small or infinitely many}\\
\hline
\end{tabular}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}