\section[The Analysis of Variance]{Models for Categorical Experiments: \\
The Analysis of Variance}

An experiment, in the sense we will use the
term, is described by the values of some chosen variables for each
of a number of runs or observations.
In a {\em designed} experiment, some of those variables will,
in principle, have
values chosen by the experimenter in advance.
In classical experimental design, these variables are categorical, and
usually called the {\em factors} in the design.
Other variables will be observed when the experimental runs
take place.
Depending on the real situation and on the experimenter's viewpoint,
such a variable may be a {\em response} whose behavior is the central
point of the experiment, or an auxiliary variable (usually called a
{\em covariate}),
possibly needed to understand the behavior of the response.
Chapter \DataChapter\ provided a natural way to represent such an experiment in S; namely,
as a data frame.
The factors  are categories -- objects representing
variables with values from a finite, specified set of levels.
The science and craft of experimental design involves choosing
the  factors to provide as much relevant information
as possible in an experiment of limited size.
The responses and covariates are treated as numeric variables.

The choice of design is inseparably tied to assumptions about the
kind of model that will adequately represent the results of the experiment.
Not only does the experimenter need to choose variables that will
prove to be important in predicting the response,
but the actual (and presumably unknown)
structural and statistical form of the model,
for example, assumptions about interactions between
factors, affect the choice of design.
So the design and the analysis are deeply interdependent.
Having said that, nevertheless one of the themes of this chapter
will be that a suitably general computational approach helps
provide useful analysis even when the experiment, as run,
turns out to depart from the assumptions made in choosing the design.

Complete factorial designs consist of one run for each possible combination
of levels for each of the factors.
For example, Table \ref{bhh.table} shows a design with 3 factors, each
having two possible levels, giving 8 runs.
In this experiment, as described in the book
{\em Statistics for Experimenters}, (Box, Hunter, and Hunter, 1978, p. 307),
the experimenter plans to run a process using two catalysts,
identified only as {\tt A} and {\tt B}, at two choices of concentration
and with two choices for the temperature at which the process runs.
Notice that two of the three variables are inherently quantitative,
even though they are to be treated as factors in the design, by
choosing two specific values.
Such variables are {\tt ordered} factors as described
in Chapter \DataChapter, although with only two levels the distinction
is not important.
\begin{table}[bht]
\begin{Example}
     Temp  Conc  Cat  Yield 
1    160   20    A    60   
2    180   20    A    72   
3    160   40    A    54   
4    180   40    A    68   
5    160   20    B    52   
6    180   20    B    83   
7    160   40    B    45   
8    180   40    B    80   
\end{Example}
\caption[$2^3$ Experimental Design]{\label{bhh.table} A factorial experiment with factors
temperature, concentration of catalyst, and catalyst type, each having
two levels.  The response is the yield of the process.}
\end{table}

The scenario is that the experimenter now runs the process
under each of the 8 conditions given by the design, and records the
yield,
as shown in the last column of Table \ref{bhh.table}.
In many situations, the experimenter will choose to randomize the order of
the runs, to the extent practicable, in the hope of disentangling
sequential effects from the factors of interest.
The results of the experiment are now available for analysis.
As always, some careful studies of the data, particularly through
plots, should precede or at least accompany the formal modeling.
One such graph is shown in Figure \ref{bhh.figure}: the average of the
values of {\tt Yield} is shown for each level of each factor.
\begin{figure}
\psfig{figure=bhhyield.p,width=3in,silent=}
\caption[Plot of Yield]{\label{bhh.figure}Yield from the experiment in Table \ref{bhh.table}, averaged for each level of the factors.}
\end{figure}

The analysis of variance, the principal classical model for categorical
experiments,
uses formulas and estimation that are formally special cases of the linear models
discussed in Chapter \LinearChapter.
The emphases differ substantially, however.
New
questions arise, such as which factors should be assumed to interact.
Discussion centers more on the contribution of terms in the model
to the total variation of the response
and less on individual coefficients.
The categorical nature of the variables means that detailed study of
the way in which the response depends on a single predictor variable
is usually impossible -- with only two levels of temperature, the
form of dependence of yield on temperature will not be available.
The compensation is that much broader exploration of the
effects of several factors may be possible.

The experiment in Table \ref{bhh.table} is small and complete.
Such experiments tend perhaps to appear mostly in textbooks.
In practice, a large number of factors, more factor levels,
incomplete designs, and other practicalities are likely to
arise.
In choosing examples for this Chapter, we will mix small examples for ease of
illustration with more realistic examples to emphasize the issues such
experiments raise.

An interesting, real experiment on the quality of integrated circuit fabrication
was reported by Phadke, Kackar, Speenyey and Grieco in {\em Bell System
Technical Journal}, (1983), vol. 62, pp. 1273-1309.
The objective of the experiment was to investigate various factors that
might affect the quality of the fabrication process, with the goal of
controlling both the average level and the variability of the quality.
The experimental design had 18 runs, varying 9 factors based on an
orthogonal array design (p. \pageref{orth.array}).
In principle two wafers would have been manufactured for each run, but 3 wafers
ended up missing.
Three different measures of quality were used, called pre-etch line width, post-etch line
width, and post-etch window size.
Window size is an interesting variable, combining categorical and numeric data,
but its analysis goes somewhat beyond what we can discuss in this chapter.
The two line widths are numeric responses, measured five times on each wafer.
The published analysis uses the mean and standard deviation for each experimental
run for pre- and post-etch line width.
Table \ref{wafer.table} shows a sample of 6 runs to suggest the form of the data.
\begin{table}[bht]
\begin{Example}
   maskdim  visc.tem   spinsp baketime 
 2       2    204,90   normal       30
 5       2    206,90   normal       30
10     2.5    204,90      low       40
14     2.5    206,90   normal       40
15     2.5    206,90     high       20
18     2.5   204,105     high       30

   aperture  exptime devtime etchtime 
 2        2   normal      45     13.2
 5        3     -20%      30     14.5
10        3   normal      45     14.5
14        1   normal      30     15.8
15        2     -20%      45     14.5
18        1   normal      60     14.5

   pre.mean pre.dev post.mean post.dev  N 
 2    2.684  0.1196      3.14    0.063 10
 5    1.870  0.1168      1.72    0.400  5
10    2.660  0.1912      3.31    0.350 10
14    2.308  0.0964      3.14    0.160 10
15    2.464  0.0385      2.55    0.210  5
18    3.494  0.0473      4.34    0.078  5
\end{Example}
\caption[Wafer etch experiment]{\label{wafer.table} Part of an experiment (6 of 18 runs) on
 wafer fabrication: 8 factors, 4 responses and the number of measurements per run.}
\end{table}
This experiment will be used several times in the chapter to illustrate some
interesting techniques of analysis.
See the reference for a fuller discussion
of the experiment and additional analysis.


Models for experiments differ from other linear models also in the emphasis given
to structure among factors and in the error assumptions.
Factors may be dependent on other factors, in that levels of the
one factor are only meaningful within each level of the other factors,
giving rise to {\em nested} or {\em hierarchical} models.
The models of Chapter \LinearChapter\ may also be extended by assuming
that the given set of levels for some factors is only a representative sample of
possible levels, giving rise to {\em random effects} terms.
The experimental situation may also imply that the errors of observation
depend on some factors, giving rise to an {\em error model};
in our approach, to a specification of the appropriate terms in
the error as an additional part of the model formula.
In this case, the analysis of variance must
reflect the multiple sources of error, so that estimated effects
can be compared to the appropriate error estimates.
These aspects of factorial experiments will be handled in
this chapter by extensions to the basic computational techniques
developed in earlier chapters, while keeping the
organization built around formula, data, and fitted model.
