\documentstyle[psbox,ps,11pt,fullpage]{article}

\input epsf

\begin{document}
\header{MAS.622}{2}
\problem{1.1}
\part{a}

In general, the normalization in front of a univariate Gaussian is $\frac{1}{\sqrt{2\pi}\sigma}$, so...

\par

$p_{y|\omega_1}(\hat{y}|\omega_1)$ has a variance of $\sqrt{3}$, so $k_1$ is $\frac{1}{\sqrt{6\pi}} = .2303$.

\par

$p_{y|\omega_2}(\hat{y}|\omega_2)$ has a variance of $1$, so $k_2$ is $\frac{1}{\sqrt{2\pi}} = .3989$.

\par
See plot in Figure \ref{p1pa}.

\begin{figure}[hbpt]
\begin{center}
\mbox{\epsfysize=3in \epsfbox{p1p1.eps}}
\end{center}
\caption{$p_{y|\omega_1}(\hat{y}|\omega_1)$ (blue) and $p_{y|\omega_1}(\hat{y}|\omega_2)$ (red) Decision boundaries are marked in green.  Between the lines, select 
$\omega_2$, otherwise $\omega_1$.}

\label{p1pa}
\end{figure}

\part{b}

Using the DHS notation, $C_ij = \lambda(\alpha_i|\omega_j)$:

\begin{eqnarray*}
R(\alpha_1|{\bf y}) = \lambda(\alpha_1|\omega_2)P(\omega_2|{\bf y}) = P(\omega_2|{\bf y}) = \frac{p(\hat{y}|\omega_2)}{p(\hat{y}|\omega_1) + p(\hat{y}|\omega_2)}\\
R(\alpha_2|{\bf y}) = \lambda(\alpha_2|\omega_1)P(\omega_2|{\bf y}) = \sqrt{3}P(\omega_1|{\bf y}) = \sqrt{3}\frac{p(\hat{y}|\omega_1)}{p(\hat{y}|\omega_1) + p(\hat{y}|\omega_2)}\\
\end{eqnarray*}

\part{c}

Solving for $y$ in $e^{{\frac{-y^2}{6}}} =
\sqrt{3}e^{\frac{-(y-2)^2}{2}}$ yields decision boundaries of $3 + \sqrt{3 + \frac{3}{2}\log{3}} = 5.1559$ and $3 - \sqrt{3 + \frac{3}{2}\log{3}} = 0.8441$

\begin{equation}
\begin{array}{lr}
R(\alpha_1|{\bf y}) > R(\alpha_2|{\bf y}) {\rm (choose } \omega_2 {\rm )}& 0.8441 < y < 5.1559\\
R(\alpha_1|{\bf y}) < R(\alpha_2|{\bf y}) {\rm (choose } \omega_1 {\rm )}& -\infty < y < 0.8441\\
R(\alpha_1|{\bf y}) < R(\alpha_2|{\bf y}) {\rm (choose } \omega_1 {\rm )}& 5.1559 < y < \infty\\
\end{array}
\end{equation}

\part{d}

Using {\tt erf()}, the probability of $\omega_2$ being the actual state
in the $ y < .8441 $ region is $\frac{.1239}{.1239+.6870} = .1528$, and
the probability of $\omega_2$ being the actual state in the $ y >
5.1559$ region is $\frac{.00080002}{.0015+.00080002} = .3478$. and the
probability of $\omega_1$ being the actual stat in the intermediate
region is $\frac{.3415}{.3415+.8753} = .2807$.

\par

\begin{tabular}{||l|c|c||}\hline
{\bf Region}& {\bf Risk of choosing $\omega_1$}& {\bf Risk of choosing $\omega_2$}\\\hline
$-\infty < y < 0.8441$& .1528& 1.4674\\\hline
$0.8441 < y < 5.1559$& .7193& .4862\\\hline
$5.1559 < y < \infty$& .3478& 1.1296\\\hline
\end{tabular}

\problem{2}

\part{a}

Hans' decision rule to minimize his error rate is:\\
If $x_1 = 1$, choose $\omega_1$\\
If $x_1 = 2$, choose $\omega_2$\\
If $x_1 = 3$, choose $\omega_2$\\

\par

His error rate is $P({\bf x} \in {\cal R}_2|\omega_1)P(\omega_1) + P({\bf x} \in {\cal R}_1|\omega_2)P(\omega_2)$, so $P({\bf x} \in {\cal R}_2|\omega_1) = (.05*.3333+.05*.0552) = .0194$, $P({\bf x} \in {\cal R}_1|\omega_2) = (.09*.0909) = .0082$, and $P(\omega_1) = P(\omega_2) = .5$ so is total error rate is .0138.
\part{b}

Frans' decision rule to minimize his error rate is:\\

If $x_2 = 1$, choose $\omega_1$\\
If $x_2 = 2$, choose $\omega_2$\\
If $x_2 = 3$, choose $\omega_2$\\

Similarly, his error rate is $P({\bf x} \in {\cal R}_2|\omega_1)P(\omega_1) + P({\bf x} \in {\cal R}_1|\omega_2)P(\omega_2)$, so $P({\bf x} \in {\cal R}_2|\omega_1) = (.73*.4762+ .01*.0052) = .3477$, $P({\bf x} \in {\cal R}_1|\omega_2) = (.026*.0909) = .0024$, and $P(\omega_1) = P(\omega_2) = .5$ so is total error rate is .1751.

\part{c}

Hans' confidence in his classification as a function of $x_1$ is:\\
If $x_1 = 1$, .9091\\
If $x_1 = 2$, .5238\\
If $x_1 = 3$, .9448\\

\par

Frans' confidence in his classification as a function of $x_2$ is:\\
If $x_2 = 1$, .9091\\
If $x_2 = 2$, .5238\\
If $x_2 = 3$, .9448\\

\part{d}

This tells me that the confidence in a classification is not the same
as the error rate and one may fluctuate wildly while the other remains
the same.  This is due to the fact that the error rate is a function
of the confidence weighted by the probability of each measurement.
That is, Hans and Frans have equal confidence given a measurement, but
Frans is far more likely (77% of the time) to measure $x_2 = 2$, which
he has low confidence in, while Hans will only measure $x_1 = 2$,
which he has equally low confidence in, 5% of the time.

\par

Frans' classifier is better in that the error rate is lower.

\problem{3}

I'm not sure how to show that ${\bf x}^T{\bf y} < t$ is an optimal
decision rule, but it is independent of priors and costs, because both
priors and costs will only have the effect of changing the value of
the threshold, $t$.  A possible $y$ might be {\tt [1 0 1 0 1 0]} for
$n = 6$.

\problem{4}

The covariance of the classes was calculated:

\begin{verbatim}
>> cv = cov([classA; classB])

cv =

   87.5190   -2.4183   -1.8939
   -2.4183   27.2164   26.1460
   -1.8939   26.1460   27.7372
\end{verbatim}

and the eigenvectors and values:

\begin{verbatim}
>> [ev ew] = eig(cv)

ev =

    0.9960   -0.0888   -0.0045
   -0.0657   -0.7003   -0.7108
   -0.0600   -0.7083    0.7034


ew =

   87.7925         0         0
         0   53.3523         0
         0         0    1.3278

\end{verbatim}

The first eigenvector captures the most energy in the data, as it has
the largest eigenvalue.

\par

Transforming the data into the space defined by the eigenvectors, it
is conspicuous that the third eigenvector contains the most
discriminatory capability, however.  See figure \ref{p4}.

\begin{figure}
\begin{center}
\mbox{\epsfysize=3in \epsfbox{p4.eps}}
\end{center}
\caption{Class A is represented by the red +'s and class B by the blue x's.  The axes are labeled with the eigenvector number where axis 1 is the eigenvector with the greatest eigenvaluse and axis 3 is the eigenvector with the least eigenvalue.}

\label{p4}
\end{figure}

Clearly the Most Expressive Feature is not at all the Most
Discriminating Feature.  In fact, the Least Expressive Feature is the
Most Discriminating Feature, in this case.  The possibility of any
correlation between the expressivity and the discriminatory capability
of a given feature in a data set is going to be highly domain specific.

\problem{5}

A few definitions first:\\
$\omega_1$: The car has original paint\\
$\omega_2$: The car has already been repainted\\
$P(\omega_1) = .35$\\
$P(\omega_2) = .65$\\
\\
Assuming Billy doesn't have any cashflow or tax related issues which
would make an \$80 cost more (or less) than twice as bad as a \$40 cost:\\
$\lambda_{11} = 40$\\
$\lambda_{22} = 75$\\
$\lambda_{12} = 115$\\
$\lambda_{21} = 140$\\

\par
\part{a}
See Figure \ref{p5pa}.

\begin{figure}[htbp]
\begin{center}
\mbox{\epsfysize=3in \epsfbox{p5pa.eps}}
\end{center}
\caption{The red line is the distribution of measurements if the car has original paint.  The blue line is the distribution of measurements if the car has been repainted.}

\label{p5pa}
\end{figure}

\part{b}

See Figure \ref{roc}.

\begin{figure}[htbp]
\begin{center}
\mbox{\epsfysize=3in \epsfbox{roc.eps}}
\end{center}
\caption{The ROC curve for classification of cars as repainted}

\label{roc}
\end{figure}

\part{c}

See Figure \ref{p5pc}.

\begin{figure}[htbp]
\begin{center}
\mbox{\epsfysize=3in \epsfbox{p5pc.eps}}
\end{center}
\caption{Expected cost based agains threshold.  The red curve is the expected cost for cars classified as original paint.  The blue curve is the expected cost for cars classified as repainted, and the green is the total expected cost}

\label{p5pc}
\end{figure}

\part{d}

If the threshold is set at 4.633, it will correctly classify a car
that in fact has original paint 95% of the time.  The expected cost
will be \$68.38.  He will be 93\% confident when he classifies an
unknown car as original paint, and 94\% confident when he classifies
an unknown car as repainted.

\par

The cost will be minimized if the threshold is set to 5.0. The
expected cost will be \$66.34. He will be 90\% confident when he
classifies an unknown car as original paint, and 100\% confident when
he classifies an unknown as a repainted car.

\par

He should decline the business if someone comes in with anything other
than a white car.

\problem{6}

Wilbur hasn't specified his priors.  He needs to specify with what
probability there is an error when he suspects so, independent of the
level of murmuring.  We'll define $\omega_1$ as the state if there is
an error and $\omega_2$ as the state where there is not one.



\par

If there are often mistakes on the board, it makes sense to speak up
more often.  The probability of incorrectly speaking up $P_F$, as a
function of threshold and prior is:

\begin{equation}
P_F(P(\omega_2), t) = \frac{\int_t^\infty p(\hat{y}|\omega_2) dy P(\omega_2)}{\int_t^\infty p(\hat{y}|\omega_1) dy (1-P(\omega_2)) + \int_t^\infty p(\hat{y}|\omega_2) dy P(\omega_2)}
\end{equation}

Further, the probability of speaking up correctly $P_D$, as a function
of threshold is:

\begin{equation}
P_D(t) = \frac{\int_t^\infty p(\hat{y}|\omega_2) dy}{\int_0^\infty p(\hat{y}|\omega_2) dy}
\end{equation}

Therefore $P_D$ is larger for lower values of $t$.  Since $P_F$ is
also larger for lower values of $t$, we must apply the constraint $P_F
\leq 0.05$.  To maximize $P_D$ within the constraint we want $P_F = 0.05$
Setting the above equation to 0.05, values for $t$ can be calculated.

\par

If the professor makes a lot of mistakes ($P(\omega_1) = .8$), then
Wilbur should use $t = .78$ and $P_D = .46$.  If the professor is
typically somewhat more careful (($P(\omega_1) = .4$), then Wilbur
should use $t = 1.68$ and $P_D = .19$.  And, if the professor is very
careful and it is usually Wilbur who is confused (($P(\omega_1) =
.08$), then Wilbur should use $t = 2.70$ and $P_D = .07$.

\problem{7}

If we assume with equal probability that each expert is correct, we
should use a distribution for bass of $p(w|{\rm bass}) =
\frac{e^\frac{-(w-4)^2}{2}}{2\sqrt{w\pi}} +
\frac{e^\frac{-(w-2)^2}{2}}{2\sqrt{w\pi}}$ And, given equal priors,
our decision boundaries are when $p(w|{\rm bass}) = p(w|{\rm carp})$.
Numerically finding these the thresholds are $w = 1.91$ and $w=4.08$.
If the weight is between these, it is classified as a carp, above 4.08
or below 1.91, a bass.

\par



\end{document}
