%
%Wyatt changes input 1-26-89
%
%
%
\title{Criteria for Robust Stability In A Class Of Lateral
Inhibition Networks Coupled Through Resistive Grids}
%
\author{John L. Wyatt, Jr.\\
David L. Standley\\
{\small \sl Department of Electrical Engineering and Computer Science,}\\
{\small \sl Massachusetts Institute of Technology, Cambridge, MA 02139, USA}}
%
\def\Communicate{Carver Mead}
%
\markboth{John L. Wyatt, Jr., and David L. Standley}{Robust Stability In A Class Of Lateral
Inhibition Networks}
%
\maketitle{wyatt}
%
\begin{abstract}
In the analog VLSI
implementation of neural systems, it is sometimes convenient to build lateral
inhibition networks by using a locally connected on-chip resistive grid to
interconnect active elements.  A serious problem of unwanted spontaneous
oscillation often arises with these circuits and renders them unusable in
practice.  This paper reports on criteria that guarantee these and certain other
systems will be stable, even though the values of designed elements in the
resistive grid may be imprecise and the location and values of parasitic
elements may be unknown.  The method is based on a rigorous, somewhat novel
mathematical analysis using Tellegen's theorem (Penfield et al.\ 1970) from
electrical circuits and the idea of a Popov multiplier (Vidyasagar 1978; Desoer
and Vidyasagar 1975) from control theory. The criteria are {\it local\/} in that
no overall analysis of the interconnected system is required for their use, {\it
empirical\/} in  that they involve only measurable frequency response data on
the individual cells, and {\it robust\/} in that they are insensitive to network
topology and to unmodelled parasitic resistances and capacitances in the
interconnect network. Certain results are robust in the additional sense that
specified {\it nonlinear\/} elements in the grid do not affect the stability
criteria. The results are designed to be applicable, with further development, to
complex and incompletely modelled living neural systems.
\end{abstract}

\section{Introduction}

In the VLSI implementation of lateral inhibition and certain other types of
networks, active cells are locally interconnected through an on-chip resistive
grid. Linear resistors fabricated in, e.g., polysilicon, could yield a very
compact realization, and nonlinear resistive grids, made from MOS transistors,
have been found useful for image segmentation (Hutchinson et al.\ 1988). 
Networks of this type can be divided into two classes: feedback systems and
feedforward-only systems.  In the feedforward case one set of amplifiers imposes
signal voltages or currents on the grid and another set reads out the resulting
response for subsequent processing, while the same amplifiers both ``write to"
the grid and ``read from" it in a feedback arrangement.  Feedforward networks of
this type are inherently stable, but feedback networks need not be.

A practical example is one of Mahowald and Mead's retina chips (Mead and
Mahowald 1988; Mead 1988) that achieve edge enhancement by means of lateral
inhibition through a resistive grid.  Figure~1a shows a single cell in an earlier
version of this chip, and figure~1b illustrates the network of
interconnected cells.  
Experiment has shown that the individual cells in this system
are open-circuit stable and remain stable when the output of amplifier
\#2 is connected to a voltage source through a resistor, but the
interconnected system oscillates so badly
that the earlier design is scarcely usable\footnote{The later design
reported in (Mead and Mahowald 1988) avoids stability problems altogether, at a
small cost in performance, by redesigning the circuits to passively sense the
grid voltage in a ``feedforward" style as described above.} (Mahowald and Mead
1988).  Such oscillations can readily occur in most resistive grid circuits with
active elements and feedback, even when each individual cell is quite stable. 
Analysis of the conditions of instability by conventional methods appears
hopeless, since the number of simultaneously active feedback loops is
enormous.

\begin{figure}
%\vskip12ptplus0ptminus0pt
%\hfil\framebox[4in]{\vbox to 2.63in{}}\hfil\vskip 8pt\hfil
%\framebox[2in]{\vbox to
%2.25in{}}\hfil\vskip 8pt\hfil
%\framebox[2in]{\vbox to 1.56in{}}\hfil\vskip12ptplus0ptminus0pt
\vspace{443pt}
 \caption{(a) This
photoreceptor and signal processor circuit, using two MOS amplifiers, realizes
spatial lateral inhibition and temporal sharpening by communicating with similar
cells through a resistive grid. The resistors will often be nonlinear by design.
(b)  Interconnection of cells through a hexagonal
resistive grid.  Cells are drawn as 2-terminal elements with the power
supply and signal output lines suppressed.  The voltage on the
capacitor in any given cell is affected both
by the local light intensity incident on that cell and by the capacitor voltages
on neighboring cells of identical design.  The necessary ingredients for
instability --- active elements and signal feedback --- are both present in
this system. (c) Grid resistors with a nonlinear characteristic of the form
$i=tanh(v)$ can be useful in image segmentation (Hutchinson et
al.\ 1988).} \end{figure}

This paper reports a practical design approach that rigorously
guarantees such a system will be stable if the active cells meet
certain criteria.  The work begins with the
naiv\'{e} observation that the system would be stable if we could
design each individual cell so that,
although internally active, it acts like a passive system as seen from
the resistive grid.  The design goal in that case would be 
that each cell's output impedance should be a {\it positive-real\/}
(Vidyasagar 1978; Desoer and Vidyasagar 1975; Anderson and Vongpanitlerd
1973) function.  This is sometimes possible in practice; we will show that
the original network in figure~1a would satisfy this condition in the absence of
certain parasitic elements.  Furthermore, it is a condition one can verify
experimentally by frequency-response measurements.

It\looseness=2~is obvious that a collection of cells that appear
passive at their terminals will form a stable system when
interconnected through a passive medium such as a resistive grid, and
that the stability of such a system is {\it robust\/} to perturbations by
passive parasitic elements in the network.  The work reported here goes
beyond that observation to
provide (i) a
demonstration that the passivity or positive-real condition is much
stronger than we actually need and that weaker conditions, more easily
achieved in practice, suffice to guarantee robust stability of the linear active
network model, and (ii) an extension of the analysis to the {\it nonlinear\/}
domain that furthermore rules out sustained {\it large-signal\/} oscillations
under certain conditions.  A key feature of the integrated circuit environment
that makes these results applicable is the almost total absence of on-chip
inductance.  While the {\it cells\/} can appear inductive, as in figure~3c,
the absence of inductance in our {\it grid models\/}
makes these theorems possible.

Note that these results do not apply directly to networks
created by interconnecting neuron-like elements, as conventionally
described in the literature on artificial neural systems.
The ``neurons" in, e.g., a Hopfield network (Hopfield 1984) are
{\it unilateral 2-port elements\/} in which the input and output are
both voltage signals.  The input voltage uniquely and instantaneously
determines the output voltage of such a neuron model, but the output
can only affect the input via the resistive grid.  In contrast, the
cells in our system are {\it 1-port electrical elements\/} (temporarily
ignoring the optical input channel) in which the port voltage and port
current are the two relevant signals, and each signal affects the
other through the cell's internal dynamics (modeled as a Thev\'{e}nin
equivalent impedance) as well as through the grid's response.

It is apparent that uncontrolled spontaneous oscillation is a
potential problem in living neural systems, which typically also
consist of active elements arranged in feedback loops.  Biological
systems have surely solved {\it the same problem\/} we attack in
this paper.  It is reasonable to believe that stability has 
strongly constrained the set of network configurations nature has
produced.  Whatever Nature's solutions may be, we suspect they have at
least three features in common with the ones proposed here: (1)
{\it robustness\/} in the face of wide component variation and the
presence of parasitic network elements, (2) reliance on {\it
empirical\/} data rather than anything we would recognize as a theory or
analytic method, (3) stability strategies based on predominantly
{\it local\/} information available to each network element.

Several reports on this work have appeared and will appear in (Wyatt and
Standley 1988; Standley 1989; Standley and Wyatt 1989; 1988a; 1988b) during its
development; a longer tutorial exposition will be given in the second printing
of (Mead 1988).

\section{The Linear Theory}

\subsection{Terminology}

The {\it output impedance\/} of a linear system is a measure of
the voltage response due to a change in output current 
while the input (light
intensity in this case) is held constant.  This standard
electrical engineering concept will play a key role here.  Figure~2a illustrates
one experimental method for measuring the output impedance, and figure~2b is a
standard graphical representation of an impedance, known as a {\it Nyquist
diagram}.  Similar plots have been used in experimental physiology (Cole 1932).

\begin{figure}%fig 2
%\vskip 12ptplus0ptminus0pt\hfil\framebox[5.38in]{\vbox to 2.63in{}}\hfil\vskip 8pt
%\hfil\framebox[4.38in]{\vbox to 3.5in{}}\hfil\vskip12ptplus0ptminus0pt
\vspace{443pt}
\caption{(a) Simplified experimental measurement of the output
impedance of a cell.  A sinusoidal current $i = A cos(\omega t)$ is
injected into the output and the voltage response $v = B cos( \omega t
+ \phi )$ is measured.  The impedance, which has magnitude B/A and phase
$\phi$, is typically treated as a complex number $Z(i \omega )$ that
depends on the frequency $\omega$.  (b) Example of the {\it Nyquist diagram\/}
of an impedance.  This is a plot in the complex plane of the value of the
impedance, measured or calculated at purely sinusoidal frequencies, ranging
from zero upward toward infinity.  It is not essential to think of Nyquist 
diagrams as representing complex numbers: they are simply polar plots in
which radius represents impedance magnitude and angle to the horizontal axis
represents phase.  The diagram shown here is the Nyquist plot of the 
positive-real impedance in equation (2.1).}
\end{figure}

\begin{figure}
\vskip12ptplus0ptminus0pt
\hfil\framebox[4.13in]{\vbox to 2.88in{}}\hfil\vskip8pt
\hfil\framebox[4.13in]{\vbox to 2.38in{}}\hfil\vskip12ptplus0ptminus0pt
\caption{(a) Elementary model for an MOS amplifier.  These
amplifiers have a relatively high
output resistance, which is determined by a bias setting (not shown). (b)
Linearity allows this simplification of the network topology for the circuit in
figure~1a without loss of information relevant to stability.  The capacitor 
in figure~1a has been
absorbed into the output capacitance of amp \#2. (c) Passive network 
realization of the output impedance given
in equation (2.1) for the network in (b).}
\end{figure}

In the context of this work, an impedance is said to be {\it
pos\-i\-tive-real\/} (Vidyasagar 1978, Desoer and Vidyasagar 1975, Anderson
and Vongpanitlerd 1973) if it is stable (i.e., has no poles or zeroes in
the right-half plane) and its Nyquist diagram lies entirely in the right-half
plane (i.e., in the language of complex numbers, $Re \{ Z(i \omega ) \} \geq 0$
for all purely sinusoidal frequencies $\omega$).  Figure 2a is an example, while
the system represented in figure~4 is stable but not positive-real.

\begin{figure}
\figspace{5.38in}{3.82in}
\caption{Nyquist diagram of an impedance that satisfies the
{\it Popov criterion}, defined as follows.  A linear impedance $Z(s)$ satisfies
the Popov criterion if $(1+ \tau s)Z(s)$ is positive-real for some
$\tau > 0$.  The ``Popov multiplier'' $(1+ \tau s)$ modifies the Nyquist
diagram by stretching and rotating it counterclockwise for $\omega > 0$.
The impedance plotted here is active and thus is not positive-real, but the 
rotation due to the $(1+\tau s)$ term can make it positive-real for an appropriate
value of $\tau$.  The Popov criterion is a condition on the linear elements
that is weaker than passivity: active elements satisfying this criterion are
shown to pose no danger of instability even when nonlinear resistors and capacitors
are present in the grid.}
\end{figure}

A deep link between positive-real functions, physical networks and
passivity is established by the classical result in linear circuit
theory which states that $H(s)$ is positive-real if and only if it is
possible to synthesize a 2-terminal network of positive linear
resistors, capacitors, inductors and ideal transformers that has
$H(s)$ as its driving-point impedance (Anderson and Vongpanitlerd 1973). 

This work was originally motivated by the following linear analysis of a  model for the circuit
in figure~1a.  For an initial approximation to the output impedance of
the cell we use the elementary model shown in figure~3a for the
amplifiers and simplify the circuit topology within a
single cell as shown in figure~3b.

Straightforward calculations show that the output impedance is given by

\begin{equation}
Z^{-1}(s) = [g_{m_{2}} + R_{o_{2}}^{-1} + s C_{o_{2}}] + \frac{g_{m_{1}} g_{m_{2}}
R_{o_{1}}}{(1+s R_{o_{1}} C_{o_{1}})}.
\end{equation}

This is a positive-real impedance that could be realized by a passive network
of the form shown in figure~3c, where

\begin{equation}
R_1 = (g_{m_{2}} + R_{o_{2}}^{-1})^{-1}, R_2 = (g_{m_{1}} g_{m_{2}}
R_{o_{1}})^{-1}, \: {\rm and} \: L = C_{o_{1}}/g_{m_{1}} g_{m_{2}} \: .
\end{equation}

Of course this model is oversimplified, since the circuit {\it does\/}
oscillate. Transistor parasitics and layout parasitics cause the output
impedance of the individual active cells to deviate from the form
given in equations (2.1) and (2.2), and any very accurate model will
necessarily be quite high order.  The following theorem shows 
how far one can relax the positive-real condition and still guarantee that the entire network
is robustly stable.  It obviously applies to a much wider range of linear networks
than has been discussed here.

A linear network is said to be {\it stable\/} if for any initial
condition the transient response converges asymptotically to a constant.

\newtheorem{theorem}{Theorem}
\begin{theorem}
Consider the class of linear networks of {\it
arbitrary\/} topology, consisting of any number of positive 2-terminal resistors
and capacitors and of $N$ lumped linear impedances $Z_n (s), n=1,2,\ldots, N$,
that are open- and short-circuit stable in isolation, i.e., that have no poles or
zeroes in the closed right-half plane.  Every such network is {\it
stable\/}
if at each frequency
$\omega \geq 0$ there exists a phase angle $\theta (\omega)$ such that
$0 \geq \theta (\omega) \geq -90^\circ$ and $| \angle Z_n (i \omega ) -
\theta (i \omega ) | < 90^\circ , n=1,2,\ldots, N$.
\end{theorem}

An equivalent statement of this last condition is that the Nyquist
plot of each cell's output impedance for $\omega \geq 0$ never
intersects the 2nd quadrant of the complex plane (figure~4 is an
example), and that no two cells' output
impedance phase angles can ever differ by as much as 180$^\circ$.  If
all the active cells are designed identically and fabricated on the
same chip, their phase angles should track fairly closely in practice, and
thus this second condition is a natural one.

The theorem is intuitively reasonable and serves as a practical design
goal.  The assumptions guarantee that
the cells cannot resonate with one another at any purely sinusoidal
frequency $s = j \omega$ since their phase angles can never differ by
as much as 180$^\circ$, and they can never resonate with the
resistors and capacitors since they can never appear simultaneously active
and inductive at any sinusoidal frequency.  A more advanced argument (Standley
and Wyatt 1989) shows that exponentially growing instabilities are also ruled
out.

\section{Stability Result for Networks with Nonlinear
Resistors and\hfil\break Capacitors}

The previous results for linear networks can afford some limited
insight into the behavior of nonlinear networks.  If a linearized model is stable, then
the equilibrium point of the original nonlinear network must be {\it
locally stable}.  But
the result in this section, in contrast, applies to the full
nonlinear circuit model and allows one to conclude that in certain
circumstances the network cannot oscillate or otherwise fail to
converge {\it even if\/} the initial
state is {\it arbitrarily far from\/} the equilibrium point.
Figure~4 introduces the {\it Popov criterion}, which is the basis of the
following theorem.  This is the first nonlinear result of its type that
requires no assumptions on the network topology.

\begin{theorem}
Consider {\em any\/} network consisting of nonlinear resistors and
capacitors and linear active cells with output impedances $Z_n(s),
n=1,2,\ldots, N$.  Suppose
\begin{enumerate}
\begin{enumerate}
\leftskip 36pt
\item the nonlinear resistor and capacitor
characteristics, $i_j = g_j (v_j)$ and $q_k = h_k (v_k)$, respectively, are
monotone increasing continuously differentiable functions, and
\leftskip 36pt
\item the impedances $Z_n (s)$ all satisfy the Popov
criterion for some common value of $\tau >0$.
\leftskip 36pt
\end{enumerate}
\end{enumerate}
\end{theorem}

Then the network is stable in the sense that, for any initial
condition at $t=0$,
\begin{equation}
\begin{array}{rcl}
{\displaystyle \int^\infty_0 [ } & 
{\displaystyle
\sum_{\stackrel{\mbox{\tiny all~resistors}}{\mbox{\tiny
and~capacitors}}} i^2_k (t)} &  {\displaystyle ] dt < \infty} \end{array}
\end{equation}

\section*{Acknowledgments}
We sincerely thank Professor Carver Mead of Cal Tech
for encouraging this work, which was supported by Defense Advanced Research
Projects Agency (DARPA) Contract No.\ N00014-87-K-0825 and National
Science Foundation (NSF) Contract No.\ MIP-8814612.

\begin{thebibliography}

\indent\indent
Anderson, B.D.O.\ and S. Vongpanitlerd. 1973. {\it Network
Analysis and Synthesis --- A Modern Systems Theory Approach},
Englewood Cliffs, NJ: Prentice-Hall.

Cole, K.S. 1932. Electric Phase Angle of Cell Membranes. {\it J.
General Physiology} {\bf 15:6}, 641--649.

Desoer, C.\ and M. Vidyasagar. 1975. {\it Feedback Systems:
Input-Output Properties}. New York: Academic Press.

Hopfield, J.J. 1984. Neurons with Graded Response have
Collective Computational Properties Like Those of Two-state Neurons.
{\it Proc. Nat'l. Acad. Sci.} {\bf 81}, 3088--3092.

Hutchinson, J., C.~Koch, J.~Luo, and C.~Mead. 1988. Computing
Motion Using Analog and Binary Resistive Networks.
{\it Computer\/} {\bf 21:3}, 52--64.

Mahowald, M.A.\ and C.A. Mead. 1988. Personal communication.

Mead, C.A. 1988. {\it Analog VLSI and Neural Systems}.
Addison-Wesley.

Mead, C.A.\ and M.A. Mahowald. 1988. A Silicon Model of Early
Visual Processing. {\it Neural Networks\/} {\bf 1:1}, 
91--97.

Penfield, P., Jr., R.~Spence, and S.~Duinker. 1970.
{\it Tellegen's Theorem and Electrical Networks}. Cambridge, MA: MIT Press.

Standley, D.L. 1989. Design Criteria
Extensions for Stable Lateral Inhibition Networks in the Presence of
Circuit Parasitics. {\it Proc.\ 1989 IEEE Int.\ Symp.\ on Circuits and
Systems}, Portland, OR.

Standley, D.L.\ and J.L.~Wyatt, Jr. 1989. Stability Criterion for Lateral
Inhibition Networks that is Robust in the Presence of Integrated Circuit
Parasitics. {\it IEEE Trans. Circuits and Systems}, to appear.

\bibline ~1988a. Stability Theorem
for Lateral Inhibition Networks that is Robust in the Presence of
Circuit Parasitics. {\it Proc. IEEE Int. Conf. on Neural Networks} {\bf I},
San Diego, CA, 27--36.

\bibline ~1988b. Circuit Design
Criteria for Stability in a Class of Lateral Inhibition Neural
Networks. {\it Proc. IEEE Conf. on Decision and Control}, Austin, TX, 328--333.

Vidyasagar, M. 1978. {\it Nonlinear Systems Analysis}.
Englewood Cliffs, NJ: Prentice-Hall.

Wyatt, J.L., Jr.\ and D.L.~Standley. 1988. A Method for the
Design of Stable Lateral Inhibition Networks that is Robust in the
Presence of Circuit Parasitics. {\it In:\/} Neural Information Processing
Systems, ed.\ D. Anderson, American Institute of Physics, New York, 860--867.

\end{thebibliography}
{\flushleft
\rule{2in}{.1mm}\hfill\break
\footnotesize Received 30 September; accepted 13 October 1988.
\normalsize}
\endpaper{wyatt}





