\magnification\magstep1
\baselineskip=2\baselineskip
\parskip=6pt plus 2pt minus 1pt

\font\lf=pbkd at 20pt
%\font\lf=cmr10 at 20.736pt

\newbox\foobox

\def\problem#1{\vskip20pt\noindent\hbox to\parindent{\bf #1.\hss}}
\def\ul#1{\setbox\foobox=\hbox{#1}\lower.5pt\hbox to 0pt{\vrule height0pt
depth.6pt width\wd\foobox \hss}\copy\foobox}

\line{\hss Albert Dvornik}

\centerline{\lf Minds and Machines}
\centerline{\lf Problem Set 10a}

\vskip20pt

\problem 1
{\bf the computer model of the mind}

The computer model of the mind certainly does imply that there is a
{\it computational\/} level at which human brain can be (successfully?)
modelled.  Therefore, a succesfull computer model of the mind is
anti-physiological, at least in the sense that it declines the need
for considering the physiological factors in modelling a mind.  That
does not mean that an analysis of human mind from the neuroscientific
point of view is useless, but a ``general'' model of a thinking system
would then necessarily be computational, not physiological.

\problem 2
{\bf nudging mines and rocks}

The appropriate training `nudges' must be administered to the neural
net from the outside (Churchland mentions an {\it ``auxiliary
computer,''\/} although in the context of running into mines the
phrase ``natural selection'' comes to mind semi-seriously).  The
backward propagation mechanism involves comparing the output of the
net with the intended result, and then using some appropriately chosen
rule (the generalized delta rule in Churchland's example) to weaken or
reinforce the neural connections that had most weight in determining
the net output.  This has the effect of {\it training\/} the network
into a behavior we consider beneficial.

This method of reinforcement backs the neural links that extract some
pattern in the original input and use it to generate a specific output
we are striving for.  This effectively means that we are teaching the
net in some specific instance of pattern recognition.  In fact, this
approach seems ideal for modelling exactly the approach Churchland is
trying to back away from: the conceptual role semantics.  It seems
natural that the conceptual syntactic role of a word could be modelled
by a set of net relationships between it and nodes corresponding to
other words (and syntactic entities in general).

\problem 3{\bf Q:}
Does Churchland think that back propagation is neurophysiologically
realistic?

\indent{\bf A:}
I'm not sure Churchland is leaning either way.

In {\it ``Cognitive Activity in Artificial Neural Networks,''\/} he
does propose several ways to blend the concept with the known facts
about the physiology of the brain.  He also proposes several
alternative weight-shifing schemes (Hebbian and anti-Hebbian), and
concludes that we don't know what learning strategy (or straregies)
the brain actually uses.

But he certainly does seem to believe that the brain does use {\it
some\/} weight-shifting strategy.

\problem 4{\bf Q:}
Name one property of neural networks that Churchland describes that is
known to be neurophysiologically unrealistic?

\indent{\bf A:}
The property I have most trouble with is the geometry of Churchland's
nets.  Despite Churchland's efforts to prove the opposite (at least on
the small scale), it seems tome that the configuration of the
described nets differs a lot from the configuration of the brain.

The neurons in the brain are too often not sectioned into neat
``layers''.  The idea that each neuron in a layer connects directly to
all the neurons in an adjacent layer works only for rather small
directed groups of neurons.  And finally, neural structures often have
circular links, which disqualify the neatly organized ``directed flow
of information'' scheme.

All this seems to generate a serious problem, since bringing enough
chaos into the net layout completely eliminates the idea of simple
back-propagating learning, and causes problems for virtually any
concievable weight-shifting algorithm.

\problem 5 {\bf singer's fingers}

Since NETtalk uses experience from past words to determine
pronounciation for new ones, it is unlikely that it would be able to
determine correct pronounciations of new `nger' words if the learning
sample was equally divided between verbs (with soft `ng's) and other
words (with hard ones).  The result might be that all the `nger' words
get pronounced with `ng's between hard and soft (as a result of being
nudged in both directions), or the system might try to find a
``pattern'' and pronounce the words with either hard or soft `ng's
seemingly at random.

Churchland would probably remark that the number of nodes in the
NETtalk network is extremely small compared to the number of neurons
in the brain, and that increasing the number of neurons and hidden
layers would probably give much better performance.

\par\vskip 0pt plus\baselineskip\break
\problem 6 {\bf the Prince and the Pinker}

A particularly convincing criticism of the ``past tense network'' is
the idea that joining heterogenous types of past tense form generation
(regular and irregular) into one mixed type-- something the advocates
of neural networks are proud of-- doesn't have to be such a good idea
after all.  Prince and Pinker point out several examples of
fundamental linguistical and psychological differences between regular
and irregular verbs, and suggest that the two classes shouldn't have
been joined.

Whether they should have been joined or not is actually not the major
point here.  What is important to realize is that the past-tense
network {\it cannot\/} distinguish between regular and irregular
verbs.  The neural net structure with strictly directed information
flow does not allow for separation of the input into subclasses
(unless we explicitly teach it to do so).  A major component of human
thought process-- ability to classify objects into categories and deal
with them separately, applying Occam's razor-like rules along the
way-- is lost in the neural-net approach.

\end
