Matthew Gray

MAS.622 Problem Set 2

February 25, 1998

Problem 1.1

Part a

In general, the normalization in front of a univariate Gaussian is \frac{1}{\sqrt{2π}σ}, so... p_{y|ω_{1}}(ŷ|ω_{1}) has a variance of \sqrt{3}, so k_{1} is \frac{1}{\sqrt{6π}} = .2303. p_{y|ω_{2}}(ŷ|ω_{2}) has a variance of 1, so k_{2} is \frac{1}{\sqrt{2π}} = .3989. See plot in Figure 1.

Figure 1: p_{y|ω_{1}}(ŷ|ω_{1}) (blue) and p_{y|ω_{1}}(ŷ|ω_{2}) (red) Decision boundaries are marked in green. Between the lines, select ω_{2}, otherwise ω_{1}.

Part b

Using the DHS notation, C_{i}j = λ(α_{i}|ω_{j}):

R(α_{1}|y) = λ(α_{1}|ω_{2})P (ω_{2}|y) = P (ω_{2}|y) = \frac{p(ŷ|ω_{2})}{p(ŷ|ω_{1}) + p(ŷ|ω_{2})}

R(α_{2}|y) = λ(α_{2}|ω_{1})P (ω_{2}|y) = \sqrt{3}P (ω_{1}|y) = \sqrt{3}\frac{p(ŷ|ω_{1})}{p(ŷ|ω_{1}) + p(ŷ|ω_{2})}

Part c

Solving for y in e^{\frac{−y^{2}}{6}} = \sqrt{3}e^{\frac{−(y−2)^{2}}{2}} yields decision boundaries of 3 + \sqrt{3 + \frac{3}{2} log 3} = 5.1559 and

3 − \sqrt{3 + \frac{3}{2} log 3} = 0.8441

R(α_{1}|y) > R(α_{2}|y)(chooseω_{2}) 0.8441 < y < 5.1559
R(α_{1}|y) < R(α_{2}|y)(chooseω_{1}) −∞ < y < 0.8441
R(α_{1}|y) < R(α_{2}|y)(chooseω_{1})  5.1559 < y < ∞

(1)

Part d

Using erf(), the probability of ω_{2} being the actual state in the y < .8441 region is \frac{.1239}{.1239+.6870} = .1528, and the probability of ω_{2} being the actual state in the y > 5.1559 region is \frac{.00080002}{.0015+.00080002} = .3478. and the probability of ω_{1} being the actual stat in the intermediate region is \frac{.3415}{.3415+.8753} = .2807.

Region  Risk of choosing ω_{1} Risk of choosing ω_{2}
−∞ < y < 0.8441  .1528  1.4674
0.8441 < y < 5.1559  .7193  .4862
5.1559 < y < ∞  .3478  1.1296

Problem 2

Part a

Hans’ decision rule to minimize his error rate is: If x_{1} = 1, choose ω_{1} If x_{1} = 2, choose ω_{2} If x_{1} = 3, choose ω_{2}

His error rate is P (x ∈ R_{2}|ω_{1})P (ω_{1}) + P (x ∈ R_{1}|ω_{2})P (ω_{2}), so P (x ∈ R_{2}|ω_{1}) = (.05 ∗ .3333 + .05 ∗ .0552) = .0194, P (x ∈ R_{1}|ω_{2}) = (.09 ∗ .0909) = .0082, and P (ω_{1}) = P (ω_{2}) = .5 so is total error rate is .0138.

Part b

Frans’ decision rule to minimize his error rate is:

If x_{2} = 1, choose ω_{1} If x_{2} = 2, choose ω_{2} If x_{2} = 3, choose ω_{2}

Similarly, his error rate is P (x ∈ R_{2}|ω_{1})P (ω_{1}) + P (x ∈ R_{1}|ω_{2})P (ω_{2}), so P (x ∈ R_{2}|ω_{1}) = (.73 ∗ .4762 + .01 ∗ .0052) = .3477, P (x ∈ R_{1}|ω_{2}) = (.026 ∗ .0909) = .0024, and P (ω_{1}) = P (ω_{2}) = .5 so is total error rate is .1751.

Part c

Hans’ confidence in his classification as a function of x_{1} is: If x_{1} = 1, .9091 If x_{1} = 2, .5238 If x_{1} = 3, .9448

Frans’ confidence in his classification as a function of x_{2} is: If x_{2} = 1, .9091 If x_{2} = 2, .5238 If x_{2} = 3, .9448

Part d

This tells me that the confidence in a classification is not the same as the error rate and one may fluctuate wildly while the other remains the same. This is due to the fact that the error rate is a function of the confidence weighted by the probability of each measurement. That is, Hans and Frans have equal confidence given a measurement, but Frans is far more likely (77he has low confidence in, while Hans will only measure x_{1} = 2, which he has equally low confidence in, 5 Frans’ classifier is better in that the error rate is lower.

Problem 3

I’m not sure how to show that x^{T} y < t is an optimal decision rule, but it is independent of priors and costs, because both priors and costs will only have the effect of changing the value of the threshold, t. A possible y might be [1 0 1 0 1 0] for n = 6.

Problem 4

The covariance of the classes was calculated:

>> cv = cov([classA; classB])
cv =
   87.5190   -2.4183   -1.8939
   -2.4183   27.2164   26.1460
   -1.8939   26.1460   27.7372

and the eigenvectors and values:

Figure 2: Class A is represented by the red +’s and class B by the blue x’s. The axes are labeled with the eigenvector number where axis 1 is the eigenvector with the greatest eigenvaluse and axis 3 is the eigenvector with the least eigenvalue.

>> [ev ew] = eig(cv)
ev =
    0.9960   -0.0888   -0.0045
   -0.0657   -0.7003   -0.7108
   -0.0600   -0.7083    0.7034

ew =
   87.7925         0         0
         0   53.3523         0
         0         0    1.3278

The first eigenvector captures the most energy in the data, as it has the largest eigenvalue. Transforming the data into the space defined by the eigenvectors, it is conspicuous that the third eigenvector contains the most discriminatory capability, however. See figure 2. Clearly the Most Expressive Feature is not at all the Most Discriminating Feature. In fact, the Least Expressive Feature is the Most Discriminating Feature, in this case. The possibility of any correlation between the expressivity and the discriminatory capability of a given feature in a data set is going to be highly domain specific.

Problem 5

A few definitions first: ω_{1}: The car has original paint ω_{2}: The car has already been repainted P (ω_{1}) = .35 P (ω_{2}) = .65

Assuming Billy doesn’t have any cashflow or tax related issues which would make an $80 cost more (or less) than twice as bad as a $40 cost: λ_{11} = 40 λ_{22} = 75 λ_{12} = 115 λ_{21} = 140

Part a

See Figure 3.

Figure 3: The red line is the distribution of measurements if the car has original paint. The blue line is the distribution of measurements if the car has been repainted.

Part b

See Figure 4.

Part c

Figure 4: The ROC curve for classification of cars as repainted

See Figure 5.

Part d

If the threshold is set at 4.633, it will correctly classify a car that in fact has original paint 95will be $68.38. He will be 93% confident when he classifies an unknown car as original paint, and 94% confident when he classifies an unknown car as repainted. The cost will be minimized if the threshold is set to 5.0. The expected cost will be $66.34. He will be 90% confident when he classifies an unknown car as original paint, and 100% confident when he classifies an unknown as a repainted car. He should decline the business if someone comes in with anything other than a white car.

Problem 6

Wilbur hasn’t specified his priors. He needs to specify with what probability there is an error when he suspects so, independent of the level of murmuring. We’ll define ω_{1} as the state if there is an error and ω_{2} as the state where there is not one. If there are often mistakes on the board, it makes sense to speak up more often. The probability of incorrectly speaking up P_{F}, as a function of threshold and prior is:

P_{F} (P (ω_{2}), t) = \frac{∫_{t}^{∞} p(ŷ|ω_{2})dyP (ω_{2})}{∫_{t}^{∞} p(ŷ|ω_{1})dy(1 − P (ω_{2})) + ∫_{t}^{∞} p(ŷ|ω_{2})dyP (ω_{2})}  (2)

Further, the probability of speaking up correctly P_{D}, as a function of threshold is:

Figure 5: Expected cost based agains threshold. The red curve is the expected cost for cars classified as original paint. The blue curve is the expected cost for cars classified as repainted, and the green is the total expected cost

P_{D}(t) = \frac{∫_{t}^{∞} p(ŷ|ω_{2})dy}{∫_{0}^{∞} p(ŷ|ω_{2})dy}  (3)

Therefore P_{D} is larger for lower values of t. Since P_{F} is also larger for lower values of t, we must apply the constraint P_{F} ≤ 0.05. To maximize P_{D} within the constraint we want P_{F} = 0.05 Setting the above equation to 0.05, values for t can be calculated. If the professor makes a lot of mistakes (P (ω_{1}) = .8), then Wilbur should use t = .78 and P_{D} = .46. If the professor is typically somewhat more careful ((P (ω_{1}) = .4), then Wilbur should use t = 1.68 and P_{D} = .19. And, if the professor is very careful and it is usually Wilbur who is confused ((P (ω_{1}) = .08), then Wilbur should use t = 2.70 and P_{D} = .07.

Problem 7

If we assume with equal probability that each expert is correct, we should use a distribution for

bass of p(w|bass) = ^{\frac{e^{\frac{−(w−4)^{2}}{2}}}{2\sqrt{wπ}}} + ^{\frac{e^{\frac{−(w−2)^{2}}{2}}}{2\sqrt{wπ}}} And, given equal priors, our decision boundaries are when p(w|bass) = p(w|carp). Numerically finding these the thresholds are w = 1.91 and w = 4.08. If the weight is between these, it is classified as a carp, above 4.08 or below 1.91, a bass.