\documentclass{article}
\input{../macros.tex}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Lecture 32: Exam 3 review}

The exam will cover the material through the singular value decomposition.  Linear transformations and change of basis will be covered on the final.

The main topics on this exam are:
\begin{itemize}
\item Eigenvalues and eigenvectors
\item Differential equations $\frac{d\vb u}{dt} = \mb A \vb u$ and exponentials $e^{\mb A t}$
\item Symmetric matrices $\mb A = A^T$:
These always have real eigenvalues, and they always have ``enough'' eigenvectors.  The eigenvector matrix $\mb Q$ can be an orthogonal matrix, with $\mb A = \mb Q \mb \Lambda \mb Q^T$.
\item Positive definite matrices
\item Similar matrices $\mb B = \mb M^{-1} \mb A \mb M$.
Matrices $A$ and $B$ have the same eigenvalues; powers of $\mb A$ will ``look like'' powers of $\mb B$.
\item Singular value decomposition
\end{itemize}

\subsection*{Sample problems}

\begin{enumerate}

\item This is a question about a differential equation with a skew
  symmetric matrix.

Suppose
$$\frac{d \vb u}{dt} = \mb A \vb u = \threebythree 0{-1}010{-1}010 \vb u.$$
The general solution to this equation will look like $$\vb u(t) = c_1 e^{\lambda_1 t} \vb x_1 + c_2 e^{\lambda_2 t} \vb x_2 + c_e e^{\lambda_3 t} \vb x_3.$$

\begin{enumerate}[a)]
\item What are the eigenvalues of $\mb A$?

The matrix $\mb A$ is singular; the first and third rows are dependent, so one eigenvalue is $\lambda_1 = 0$.  We might also notice that $\mb A$ is antisymmetric ($\mb A^T = -\mb A$) and realize that its eigenvalues will be imaginary.

To find the other two eigenvalues, we'll solve the equation $|\mb A -
\lambda \mb I| = 0$.
$$\threebythreedet {-\lambda}{-1}0{1}{-\lambda}{-1}01{-\lambda}
= -\lambda^3-2\lambda = 0.$$
We conclude $\lambda_2 = \sqrt{2} \, i$ and $\lambda_3 = -\sqrt{2} \, i$.

\end{enumerate}

At this point we know that our solution will look like:
$$\vb u(t) = c_1 \vb x_1 + c_2 e^{\sqrt{2}\, i t} \vb x_2 + c_e
e^{-\sqrt{2}\, i t} \vb x_3.$$ We can now see that the solution doesn't
increase without bound or decay to zero.  The size of $e^{i\theta}$ is
the same for any $\theta$; the exponentials here correspond to points
on the unit circle.

\begin{enumerate}[a)]
\setcounter{enumii}{1}
\item The solution is periodic.  When does it return to its original value?
(What is its period?)  

This is not likely to be on the exam, but we
can quickly remark that $e^{\sqrt{2}\, it} = e^{0}$ when $\sqrt{2}t =
2\pi$, or when $t = \pi \sqrt{2}$.

\item Show that two eigenvectors of $A$ are orthogonal.  

The eigenvectors
of a symmetric matrix or a skew symmetric matrix are always
orthogonal.
One choice of eigenvectors of $\mb A$ is:
$$\vb x_1 = \threebyone 101,\; \vb x_2 = \threebyone {-1}{\sqrt{2}\,i}1,\; \vb x_3 = \threebyone 1 {\sqrt{2}i}{-1}.$$
Don't forget to conjugate the first vector when computing the inner product of  vectors with complex number entries.

\item The solution to this differential equation is $\vb u(t) = e^{\mb A t}
\vb u(0)$.  How would we compute $e^{\mb At}$?  

If $\mb A = \mb S \Lambda \mb S^{-1}$ then $e^{\mb At} = \mb S e^{\mb \Lambda t}\mb S^{-1}$ where
$$e^{\mb \Lambda t} = \threebythree {e^{\lambda_1 t}}{}{}{}\ddots{}{}{}{e^{\lambda_n t}}.$$
So $e^{\mb A t}$ comes from the eigenvalues in $\mb \Lambda$ and the
eigenvectors in $\mb S$.
\end{enumerate}
\end{enumerate}

Fact:  A matrix has orthogonal eigenvectors exactly when $AA^T = A^TA$; i.e. when $\mb A$ commutes with its transpose.  This is true of symmetric, skew symmetric and orthogonal matrices.

\begin{enumerate}
\setcounter{enumi}{1}
\item We're told that a three by three matrix $\mb A$ has eigenvalues $\lambda_1 = 0$, $\lambda_2 = c$ and $\lambda_3 = 2$ and eigenvectors
$$\vb x_1 = \threebyone 111, \vb x_2 = \threebyone 1{-1}0, \vb x_3 = \threebyone 1 1 {-2}.$$

\begin{enumerate}[a)]

\item For which $c$ is the matrix diagonalizable?

The matrix is diagonalizable if it has $3$ independent eigenvectors.
Not only are $\vb x_1$, $\vb x_2$ and $\vb x_3$ independent, they're
orthogonal.  So the matrix is diagonalizable for all values of $c$.

\item For which values of $c$ is the matrix symmetric?

If $\mb A = \mb Q \mb \Lambda \mb Q^T$ is symmetric its eigenvalues
(the entries of $\mb \Lambda$) are real.  On the other hand, if $\mb
c$ is real, then $\mb A^T = \mb Q^T \mb \Lambda^T \mb Q = \mb A$ is
symmetric.  The matrix is symmetric for all real numbers $c$.

\item For which values of $c$ is the matrix positive definite?

All positive definite matrices are symmetric, so $\mb c$ must be real.
The eigenvalues of a positive definite matrix must be positive.  The
eigenvalue $0$ is not positive, so this matrix is not positive
definite for any values of $c$.  (If $c\geq 0$ then the matrix is
positive semidefinite.)

\item Is it a Markov matrix?

In a Markov matrix, one eigenvalue is $1$ and the other eigenvalues are smaller than $1$.  Because $\lambda_3 = 2$, this cannot be a Markov matrix for any value of $c$.

\item Could $\mb P = \frac{1}{2} \mb A$ be a projection matrix?

Projection matrices are real and symmetric so their eigenvalues are
real.  In addition, we know that their eigenvalues are $1$ and $0$
because $\mb P^2 = \mb P$ implies $\mb \lambda^2 = \lambda$.  So
$\frac 12 \mb A$ could be a projection matrix if $c=0$ or $c=2$.

\end{enumerate}

Note that it was the fact that the eigenvectors were orthogonal that made it possible to answer many of these questions.

\end{enumerate}

Singular value decomposition (SVD) is a factorization
$$\mb A = \text{(orthogonal)(diagonal)(orthogonal)} = \mb U \mb \Sigma
  \mb V^T.$$ We can do this for any matrix $\mb A$.  The key is to look at
  the symmetric matrix $\mb A^T \mb A = \mb V \mb \Sigma^T \mb \Sigma
  \mb V^T$; here $\mb V$ is the eigenvector matrix for $\mb A^T\mb A$
  and $\mb \Sigma^T \mb \Sigma$ is the matrix of eigenvalues
  $\sigma_i^2$ of $\mb A^T \mb A$.  Similarly, $\mb A \mb A^T = \mb U
  \mb \Sigma \mb \Sigma^T \mb U^T$ and $\mb U$ is the eigenvector
  matrix for $\mb A \mb A^T$.  (Note that we can introduce a sign
  error if we're unlucky in choosing eigenvectors for the columns of
  $\mb U$.  To avoid this, use the formula $\mb A \vb v_i = \sigma_i
  \vb u_i$ to calculate $\mb U$ from $\mb V$.)

On the exam, you might be asked to find the SVD of a matrix $\mb A$ or
you might be given information on $\mb U$, $\mb \Sigma$ and $\mb V$
and asked about $\mb A$.

\begin{enumerate}
\setcounter{enumi}{2}

\item  Suppose $\mb \Sigma = \small \twobytwo
3002$ and $\mb U$ and $\mb V$ each have two columns.  

\begin{enumerate}[a)]
\item What can we say about $\mb A$?

We know
$\mb A$ is a two by two matrix, and because $\mb U$, $\mb V$ and $\mb
\Sigma$ are all invertible we know $\mb A$ is nonsingular.

\item What if $\mb \Sigma = \small \twobytwo 300{-5}$?

This is not a valid possibility for $\mb \Sigma$.  The singular values --
the diagonal entries of $\mb \Sigma$ -- are never negative in a
singular value decomposition.

\item What if $\mb \Sigma = \small \twobytwo 3000$?

Then $\mb A$ is a singular
matrix of rank $1$ and its nullspace has dimension $1$.  The four
fundamental subspaces associated with $\mb A$ are spanned by
orthonormal bases made up of selected columns of $\mb U$ and $\mb
V$. In this example, the second column of $\mb V$ is a basis for the
nullspace of $\mb A$.
\end{enumerate}

\item We're told that $\mb A$ is symmetric and orthogonal.  

\begin{enumerate}[a)]
\item What can we say about its eigenvalues?

The eigenvalues of symmetric matrices are real.  The eigenvalues of
orthogonal matrices $Q$ have $|\lambda| = 1$; multiplication by an
orthogonal matrix doesn't change the length of a vector.  So the
eigenvalues of $\mb A$ can only be $1$ or $-1$.

\item True or false:  $\mb A$ is sure to be positive definite.

False -- it could have an eigenvalue of $-1$, as in $\small \twobytwo 100{-1}$.

\item True or false:  $\mb A$ has no repeated eigenvalues.

False -- if $\mb A$ is a three by three matrix or larger, it's
guaranteed to have repeated eigenvalues because every $\lambda$ is $1$
or $-1$.

\item Is $\mb A$ diagonalizable?

Yes, because all symmetric and all orthogonal matrices can be diagonalized.  In fact, we can choose the eigenvectors of $\mb A$ to be orthogonal.

\item Is $\mb A$ nonsingular?

Yes; orthogonal matrices are all nonsingular.

\item Show $\mb P = \frac 12 (\mb A + \mb I)$ is a projection matrix.

We could check that $\mb P$ is symmetric and that $\mb P^2 = \mb P$:
$$\mb P^2 = \left(\frac 12 (\mb A + \mb I)\right)^2 = \frac 14 (\mb A^2 + 2 \mb A + \mb I).$$
Because $\mb A$ is orthogonal and symmetric, $\mb A^2 = \mb A^T \mb A
= \mb I$, so
$$\mb P^2 = \frac 14 (\mb A^2 + 2 \mb A + \mb I) = \frac 12 (\mb A +
\mb I) = P.$$

Or we could note that since the eigenvalues of $\mb A$ are $1$ and
$-1$ then the eigenvalues of $\frac 12 (\mb A + \mb I)$ must be $1$
and $0$.

\end{enumerate}
\end{enumerate}

These questions all dealt with eigenvalues and special matrices;
that's what the exam is about.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}