\documentclass{article}
\input{../macros.tex}

\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section*{Lecture 30: Linear transformations and their matrices}

In older linear algebra courses, linear transformations were introduced
before matrices.  This geometric approach to linear algebra
initially avoids the need for coordinates.  But eventually there must be
coordinates and matrices when the need for computation arises.

\subsection*{Without coordinates (no matrix)}

\subsubsection*{Example 1:  Projection}

We can describe a projection as a {\em linear transformation} $T$
which takes every vector in $\R^2$ into another vector in $\R^2$.  In
other words,
$$T : \R^2 \longrightarrow \R^2.$$ The rule for this {\em mapping} is
that every vector $\vb v$ is projected onto a vector $T(\vb v)$ on the
line of the projection.  Projection is a linear transformation.

\subsubsection*{Definition of {\em linear}}

A transformation $\mb T$ is {\em linear} if:
$$T(\vb v + \vb w) = T(\vb v) + T(\vb w)$$
and
$$T(c\vb v) = cT(\vb v)$$ for all vectors $\vb v$ and $\vb w$ and for
all scalars $c$.  Equivalently, $$T(c\vb v + d \vb w) = cT(\vb v) +
dT(\vb w)$$ for all vectors $\vb v$ and $\vb w$ and scalars $c$ and $d$.
It's worth noticing that $T(\vb 0) = \vb 0$, because if not it
couldn't be true that $T(c\vb 0) = cT(\vb 0)$.

\subsubsection*{Non-example 1:  Shift the whole plane}

Consider the transformation $T(\vb v) = \vb v + \vb v_0$ that
shifts every vector in the plane by adding some fixed vector $\vb v_0$ to
it.  This is {\em not} a linear transformation because $T(2\vb v) = 2\vb v +
\vb v_0 \neq 2T(\vb v)$.

\subsubsection*{Non-example 2:  $T(\vb v) = ||\vb v||$}

The transformation $T(\vb v) = ||\vb v||$ that takes any vector to its
length is not a linear transformation because $T(c \vb v) \neq c T(\vb
v)$ if $c < 0$.

\smallskip

We're not going to study transformations that aren't linear.  From
here on, we'll only use $T$ to stand for linear transformations.

\subsubsection*{Example 2:  Rotation by $45^\circ$}

This transformation $T:\R^2 \longrightarrow \R^2$ takes an input
vector $\vb v$ and outputs the vector $T(\vb v)$ that comes from
rotating $\vb v$ counterclockwise by $45^\circ$ about the origin.  Note that we can
describe this and see that it's linear without using any coordinates.

\subsubsection*{The big picture}

One advantage of describing transformations geometrically is that it
helps us to see the big picture, as opposed to focusing on the transformation's effect
on a single point.  We can quickly see how rotation by $45^\circ$ will
transform a picture of a house in the plane.  If the transformation was described in terms of a
matrix rather than as a rotation, it would be harder to guess what the
house would be mapped to.

Frequently, the best way to understand a linear transformation is to
find the matrix that lies behind the transformation.  To do this, we
have to choose a basis and bring in coordinates.

\subsection*{With coordinates (matrix!)}

All of the linear transformations we've discussed above can be
described in terms of matrices.  In a sense, linear transformations
are an abstract description of multiplication by a matrix, as in the
following example.

\subsubsection*{Example 3: $T(\vb v) = \mb A \vb v$}

Given a matrix $\mb A$, define $T(\vb v) = \mb A \vb v$.  This is a linear transformation:
$$\mb A(\vb v + \vb w) = \mb A(\vb v) + \mb A (\vb w)$$
and
$$\mb A(c\vb v) = c \mb A(\vb v).$$

\subsubsection*{Example 4}

Suppose $\mb A = \small \twobytwo 100{-1}$.  How would we describe the
transformation $T(\vb v) = \mb A \vb v$ geometrically?

When we multiply $\mb A$ by a vector $\vb v$ in $\R^2$, the $x$
component of the vector is unchanged and the sign of the $y$ component
of the vector is reversed.  The transformation $\vb v \mapsto \mb A
\vb v$ reflects the $xy$-plane across the $x$ axis.

\subsubsection*{Example 5}

How could we find a linear transformation $T: \R^3 \longrightarrow
\R^2$ that takes three
dimensional space to two dimensional space?  Choose any 2 by 3 matrix $\mb A$
and define $T(\vb v) = \mb A \vb v$.

\subsubsection*{Describing $T(\vb v)$}

How much information do we need about $T$ to to determine $T(\vb v)$ for
all $\vb v$?  If we know how $T$ transforms a single vector $\vb v_1$, we
can use the fact that $T$ is a linear transformation to calculate
$T(c\vb v_1)$ for any scalar $c$.  If we know $T(\vb v_1)$ and $T(\vb v_2)$ for two
independent vectors $\vb v_1$ and $\vb v_2$, we can predict how $T$
will transform any vector $c \vb v_1 + d \vb v_2$ in the plane spanned
by $\vb v_1$ and $\vb v_2$.  If we wish to know $T(\vb v)$ for all
vectors $\vb v$ in $\R^n$, we just need to know $T(\vb v_1), T(\vb
v_2), ..., T(\vb v_n)$ for any basis $\vb v_1, \vb v_2, ..., \vb v_n$
of the input space.  This is because any $\vb v$ in the input space
can be written as a linear combination of basis vectors, and we know
that $T$ is linear:
\begin{align*}
\vb v &= c_1 \vb v_1 + c_2 \vb v_2 + \cdots + c_n \vb v_n\\
T(\vb v) &= c_1 T(\vb v_1) + c_2 T(\vb v_2) + \cdots + c_n T(\vb v_n).\\
\end{align*}

This is how we get from a (coordinate-free) linear transformation to a
(coordinate based) matrix; the $c_i$ are our coordinates.  Once we've
chosen a basis, every vector $\vb v$ in the space can be written as a
combination of basis vectors in exactly one way.  The coefficients of
those vectors are the {\em coordinates} of $\vb v$ in that basis.

Coordinates come from a basis; changing the basis changes the
coordinates of vectors in the space.  We may not use the standard
basis all the time -- we sometimes want to use a basis of eigenvectors
or some other basis.

\subsubsection*{The matrix of a linear transformation}

Given a linear transformation $T$, how do we construct a matrix $\mb
A$ that represents it?  

First, we have to choose two bases, say $\vb v_1, \vb v_2, ..., \vb v_n$  of
$\R^n$ to give coordinates to the input vectors and $\vb w_1, \vb w_2,
..., \vb w_m$ of $\R^m$ to give coordinates to the output vectors.  We want
to find a matrix $\mb A$ so that $T(\vb v) = \mb A \vb v$, where $\vb
v$ and $\mb A \vb v$ get their coordinates from these bases.

The first column of $\mb A$ consists of the coefficients $a_{11},
a_{21}, ..., a_{1m}$ of $T(\vb v_1) = a_{11} \vb w_1 + a_{21} \vb w_2
+ \cdots + a_{1m} \vb w_m$.  The entries of column $i$ of the matrix
$\mb A$ are determined by $T(\vb v_i) = a_{1i} \vb w_1 + a_{2i} \vb
w_2 + \cdots + a_{1i} \vb w_m$.  Because we've guaranteed that $T(\vb
v_i) = \mb A \vb v_i$ for each basis vector $\vb v_i$ and because $T$
is linear, we know that $T(\vb v) = \mb A \vb v$ for all vectors $\vb
v$ in the input space.

\bigskip

In the example of the projection matrix, $n = m = 2$.  The
transformation $T$ projects every vector in the plane onto a line.  In
this example, it makes sense to use the same basis for the input and
the output.  To make our calculations as simple as possible, we'll
choose $\vb v_1$ to be a unit vector on the line of projection and
$\vb v_2$ to be a unit vector perpendicular to $\vb v_1$.  Then 
$$T(c_1 \vb v_1 + c_2 \vb v_2) = c_1 \vb v_1 + \vb 0$$ and the matrix
of the projection transformation is just $\mb A = \small
\twobytwo 1000$.
$$\mb A \vb v = \twobytwo 1000 \twobyone {c_1}{c_2} = \twobyone {c_1}0.$$

%We can do this for any linear transformation $T$; after choosing bases
%for the input and output spaces, we can always find a matrix $\mb A$
%for which $T(\vb v) = \mb A \vb v$.

This is a nice matrix!  If our chosen basis consists of
eigenvectors then the matrix of the transformation will be the
diagonal matrix $\mb \Lambda$ with eigenvalues on the diagonal.

\smallskip

To see how important the choice of basis is, let's use the standard
basis for the linear transformation that projects the plane onto a
line at a $45^\circ$ angle.  If we choose $\vb v_1 = \vb w_1 = \small \twobyone
10$ and $\vb v_2 = \vb w_2 = \small \twobyone 01$, we get the projection
matrix $P = \displaystyle{\frac{\vb a \vb a^T}{\vb a^T \vb a} = \small
  \twobytwo {1/2}{1/2}{1/2}{1/2}}.$ We can check by graphing that this
is the correct matrix, but calculating $P$ directly is more difficult
for this basis than it was with a basis of eigenvectors.

\subsubsection*{Example 6:  $\displaystyle{T = \frac{d}{dx}}$}

Let $T$ be a transformation that takes the derivative:
\begin{equation}\label{eq30.1}
T(c_1 + c_2 x + c_3x^2) = c_2 + 2c_3 x.
\end{equation} The input space is the
three dimensional space of quadratic polynomials $c_1 + c_2 x +
c_3x^2$ with basis $\vb v_1 = 1$, $\vb v_2 = x$ and $\vb v_3 = x^2$.
The output space is a two dimensional subspace of the input space with
basis $\vb w_1 = \vb v_1 = 1$ and $\vb w_2 = \vb v_2 = x$.  

This is a linear transformation!  So we can find $\mb A = \small
\twobythree 010002$ and write the transformation (\ref{eq30.1}) as a
matrix multiplication (\ref{eq30.2}): \begin{equation}\label{eq30.2}
T\left(\threebyone {c_1}{c_2}{c_3}\right) = \mb A \threebyone{c_1}{c_2}{c_3} = \twobyone{c_2}{2c_3}.\end{equation}

\subsection*{Conclusion}

For any linear transformation $T$ we can find a matrix $\mb A$ so that
$T(\vb v) = \mb A \vb v$.  If the transformation is invertible, the
inverse transformation has the matrix $\mb A^{-1}$.  The product of
two transformations $T_1:\vb v \mapsto \mb A_1 \vb v$ and $T_2:\vb w
\mapsto \mb A_2 \vb w$ corresponds to the product $\mb A_2 \mb A_1$ of
their matrices.  This is where matrix multiplication came from!

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}