\documentstyle[11pt,simplemargins]{article}
\setallmargins{1in}
% no page numbers
\pagestyle{empty}
% counter for page numbers
\newcounter{pg}
\setcounter{pg}{2}

% DEFINE THE PROBLEM SET NUMBER
\def\psnum{3}

% DEFINE THE DUE DATE
\def\due{April 1, 1992}

% title command
\newcommand{\mytitle}{{\large\noindent 6.863J Natural Language
Processing \hfill{} Juliet Bourne \\ 
Lab \#\psnum \hfill{} \due \\}
\vspace{-.43in}
\begin{center} \rule{6.5in}{.25mm} \end{center}}
% newpage setup command 
\newcommand{\mypage}[1]{\newpage{\large\noindent 6.863J Natural Language
Processing \hfill{} Juliet Bourne \\
Lab \#\psnum \hfill{} \due \\}
\vspace{-.43in}
\begin{center} \rule{6.5in}{.25mm} \end{center}}
% problem command
\newcommand{\prob}[1]{\vspace*{.15in}\noindent{\fbox{\large\bf #1}}}
% subproblem command
\newcommand{\subprob}[1]{\vspace{.15in} \noindent{\large\bf #1}}
% equations commands
\newcommand{\beq}{\vspace*{-.3in}\begin{eqnarray*}}
\newcommand{\eeq}{\end{eqnarray*}}

\begin{document}

\mytitle

\prob{Part 1}

Already handed in on March 18, 1992.

\prob{Part 2}

\subprob{Mapping}

See attached. Only LOB categories which were needed for the new
sentences were mapped to appropriate Lab 2 categories. The ``name''
category, which was not in Lab 2, was added so that proper nouns could
be handled. (Proper names in the dictionary for Lab 2 were given the
additional category of ``name''.)

\subprob{New Words Dictionary}

See attached. 

\subprob{Trace}

The trace attached is the parsing (without disambiguation) of the 7
sentences shown in the lab handout. The output from parsing and timing
all of the Lab 3 sentences (with and without disambiguation) is far too
long to print out. (These output files can be found in
\verb+/mit/jcbourne/Psets/-+ \verb+6863/lab3/{no,}dis.out+.) 

\subprob{Timing Results}

See attached for timing averages (usually over 3 parses) for each
sentence with and without using the disambiguator. From these (as shown
in the same attachment), the difference (in seconds) between not using
the disambiguator and using it is calculated. Using the program ANOVA,
the grand mean of these differences (over all sentences) and an F-test
of this grand mean (comparing it to zero) is calculated.  The grand mean
of differences is 4.39 seconds. The F-test indicates that this is
statistically significant, with the probability that this is a false
result less than 0.1 percent.

Thus it seems that using the disambiguator significantly decreases the
time it takes to parse a sentence. However, taking a look at the
differences, it can be seen that they range from 0.06 to 19.14 seconds.
For some sentences, the disambiguator seems to do no good. Yet for
others, it makes a big difference. This probably depends on the degree
of ambiguity of a sentence. 

\subprob{Other Comments}

I am surprised that the advantage in using the disambiguator was
statistically significant. (It's possible that I'm not doing the
statistics right.) Given that my grammar returns multiple parses when it
shouldn't and rejects good sentences, it is amazing that the
disambiguator can make a difference. However, it seems to in some cases.

The lab handout says ``if the word lookup can't find an LOB category for
a word in general, it will revert to the supplied lab dictionary.'' If
this is supposed to mean that words not in the LOB dictionary will be
looked up in the lab dictionary, it didn't work for me. For example,
looking up ``computed'' (a Lab 2 word) told me that it wasn't in the LOB
dictionary. 

\end{document}

