\documentclass{article}
\input{../macros.tex}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Lecture 21: Eigenvalues and eigenvectors}

The subject of eigenvalues and eigenvectors will take up most
of the rest of the course.  We will again be working with square
matrices.  Eigenvalues are special numbers associated with
a matrix and eigenvectors are special vectors.

\subsection*{Eigenvectors and eigenvalues}

A matrix $\mb A$ {\em acts on} vectors $\vb x$ like a function does, with
input $\vb x$ and output $\mb A \vb x$.  {\em Eigenvectors} are
vectors for which $\mb A \vb x$ is parallel to $\vb x$.  In other
words:
$$\mb A \vb x = \lambda \vb x.$$ In this equation, $\vb x$ is an
eigenvector of $\mb A$ and $\lambda$ is an {\em eigenvalue} of $\mb
A$.

\subsubsection*{Eigenvalue $0$}

If the eigenvalue $\lambda$ equals $0$ then $\mb A \vb x = 0\vb x =
\vb 0$.  Vectors with eigenvalue $0$ make up the nullspace of
$\mb A$; if $\mb A$ is singular, then $\lambda = 0$ is an eigenvalue of $\mb A$.

\subsection*{Examples}

Suppose $\mb P$ is the matrix of a projection onto a plane.  For any
$\vb x$ in the plane $\mb P \vb x = \vb x$, so $\vb x$ is an
eigenvector with eigenvalue $1$.  A vector $\vb x$ perpendicular
to the plane has $\mb P \vb x = \vb 0$, so this is an eigenvector with
eigenvalue $\lambda = 0$.  The eigenvectors of $\mb P$ span the whole space (but this is not true for every matrix).

\smallskip

The matrix $\mb B = \small \twobytwo 0110$ has an eigenvector $\vb x =
\small \twobyone 11$ with eigenvalue $1$ and another eigenvector $\vb
x = \small \twobyone 1{-1}$ with eigenvalue $-1$.  These
eigenvectors span the space.  They are perpendicular because $\mb B = \mb B^T$ (as we will prove).


\subsection*{$\det(\mb A - \lambda \mb I) = 0$}


An $n$ by $n$ matrix will have $n$ eigenvalues, and their sum will be
the sum of the diagonal entries of the matrix: $a_{11} + a_{22} +
\cdots +a_{nn}$.  This sum is the {\em trace} of the matrix.  For a
two by two matrix, if we know one eigenvalue we can use this fact to
find the second.

Can we solve $\mb A \vb x = \lambda \vb x$ for the eigenvalues and
eigenvectors of $\mb A$?  Both $\lambda$ and $\vb x$ are unknown; we
need to be clever to solve this problem:
\begin{eqnarray*}
\mb A  \vb x &=& \lambda \vb x\\
(\mb A - \lambda \mb I)\vb x &=& \vb 0
\end{eqnarray*}
In order for $\lambda$ to be an eigenvector, $\mb A - \lambda \mb I$
must be singular.  In other words, $\det(\mb A - \lambda \mb I) = 0$.
We can solve this {\em characteristic equation} for $\lambda$ to get
$n$ solutions.  If we're lucky, the solutions are distinct.  If not,
we have one or more {\em repeated eigenvalues}.

Once we've found an eigenvalue $\lambda$, we can use elimination to
find the nullspace of $\mb A - \lambda \mb I$.  The vectors in that
nullspace are eigenvectors of $\mb A$ with eigenvalue $\lambda$.

\subsubsection*{Calculating eigenvalues and eigenvectors}

Let $\mb A = \small \twobytwo 3113$.  Then:
\begin{eqnarray*}
\det(\mb A - \lambda \mb I) &=& \twobytwodetc {3-\lambda}11{3-\lambda}\\
&=& (3 - \lambda)^2 - 1\\
&=& \lambda^2 - 6\lambda + 8.
\end{eqnarray*}
Note that the coefficient $6$ is the trace (sum of diagonal entries) and $8$ is the determinant
of $\mb A$.  In general, the eigenvalues of a two by two matrix are
the solutions to: $$\lambda^2 - \text{trace}(\mb A)\cdot
\lambda + \det A = 0.$$

Just as the trace is the sum of the eigenvalues of a matrix, the
product of the eigenvalues of any matrix equals its determinant.

For $\mb A = \small \twobytwo 3113$, the eigenvalues are $\lambda_1 =
4$ and $\lambda_2 = 2$.  We find the eigenvector $\vb x_1 = \small \twobyone 11$ for $\lambda_1 =
4$ in the nullspace of $\mb A - \lambda_1 \mb I = \small
\twobytwo {-1}11{-1}$.

$x_2$ will be in the nullspace of $\mb A - 2 \mb I = \small \twobytwo
1111$.  The nullspace is an entire line; $x_2$ could be any vector
on that line.  A natural choice is $x_2 = \small \twobyone {-1}1$.

Note that these eigenvectors are the same as those of $\mb B = \small
\twobytwo 0110$.  Adding $3 \mb I$ to the matrix
$\mb B = \small \twobytwo 0110$ added $3$ to each of its eigenvalues and did
not change its eigenvectors, because 
$\mb A \vb x = (\mb B + 3 \mb I )\vb x = \lambda \vb x + 3 \vb x = (\lambda + 3) \vb
x$.

\subsubsection*{A caution}

Similarly, if $\mb A \vb x = \lambda \vb x$ and $\mb B \vb x = \alpha
\vb x$, $(\mb A + \mb B)\vb x = (\lambda + \alpha)\vb x$.  It would be
nice if the eigenvalues of a matrix sum were always the sums of the
eigenvalues, but this is only
true if $\mb A$ and $\mb B$ have the same eigenvectors.  The eigenvalues of the product $\mb A \mb B$ aren't usually equal to the products $\lambda(\mb A)\lambda(\mb b)$, either.

\subsubsection*{Complex eigenvalues}

The matrix $\mb Q = \twobytwo 0{-1}10$ rotates every vector in the plane by
$90^\circ$.  It has trace $0 = \lambda_1 + \lambda_2$ and determinant
$1 = \lambda_1 \cdot \lambda_2$.  Its only real eigenvector is the zero
vector; any other vector's direction changes when it is multiplied by
$\mb Q$.  How will this affect our eigenvalue calculation?
\begin{eqnarray*}
\det(\mb A - \lambda \mb I) &=& \twobytwodet {-\lambda}{-1}{1}{-\lambda}\\
&=& \lambda^2 + 1.
\end{eqnarray*}
$\det(\mb A - \lambda \mb I) = 0$ has solutions $\lambda_1 = i$
and $\lambda_2 = -i$.  If a matrix has a complex eigenvalue $a + bi$
then the {\em complex conjugate} $a - bi$ is also an eigenvalue of
that matrix.

Symmetric matrices have real eigenvalues.  For {\em antisymmetric}
matrices like $\mb Q$, for which $\mb A^T = -\mb A$, all eigenvalues are
imaginary ($\lambda = bi$).

\subsubsection*{Triangular matrices and repeated eigenvalues}

For triangular matrices such as $\mb A = \small \twobytwo 3103$, the
eigenvalues are exactly the entries on the diagonal.  In this case,
the eigenvalues are $3$ and $3$:
\begin{eqnarray*}
\det(\mb A - \lambda \det I) &=& \twobytwodetc {3-\lambda}10{3 - \lambda}\\
&=& (3 - \lambda)(3-\lambda) \quad \bigg(= (a_{11}-\lambda)(a_{22}-\lambda)\bigg)\\
&=&0,
\end{eqnarray*}
so $\lambda_1 = 3$ and $\lambda_2 = 3$.  To find the eigenvectors, solve:
$$(\mb A - \lambda \mb I)\vb x = \twobytwo 0100 \vb x = \vb 0$$ to get
$\vb x_1 = \small \twobyone 10$.  There is no independent eigenvector
$\vb x_2$.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}