\documentstyle[11pt,graphics,fullpage]{article} 
\setlength{\textheight}{10in}
\setlength{\textwidth}{6in}

\begin{document} 
 
\title {Cell Assemblies for Rapid Recognition and Disambiguation }
\author { Christian R. Huyck \\
          Middlesex University\\ 
          c.huyck@mdx.ac.uk\\} 

\date{}

\maketitle


\begin {abstract}
This document discusses reverberating circuits of neurons or Cell
Assemblies (CAs) derived from Hebb's \cite {Hebb} initial proposal.
It shows how CAs can quickly categorise an input and make a quick
decision when presented with ambiguous data.

CAs are a widely accepted theoretical construct in cognitive
neuroscience.  This document also references some initial
computational modelling of CAs.

CAs are thus an ideal mechanism for further research into both
computational and cognitive neural models.  If humans use CAs as a
basis of thought, then studying how biological systems use CAs will
provide information for computational models.  The reverse is also
true; computational modelling can direct our research activity in
biological systems.
\end{abstract}

\section {Introduction and Background} 
There are many possible connectionist architectures.  The most
successful architecture is implemented in slow moving biological
circuitry.  It exists in trillions of organisms, it can learn, be used
for language and can even exhibit consciousness.  Animal brains are
the apex of intelligence.  We have a pretty good understanding of how
neurons work (by no means complete), but we have much less
understanding of how neurons work together.

An average human neuron fires for a brief time, and then fatigues.  If
such a neuron were to represent a concept, we could only think of that
concept for a brief time. Since we can think of a given concept for at
least several seconds, a single neuron cannot represent an entire
concept.  A concept must be represented by a collection of neurons.
How might such a concept be represented?

D. O. Hebb proposed a solution to this problem \cite {Hebb}, a
reverberating circuit of cells. Hebb called this reverberating circuit
a {\em Cell Assembly} (CA). Some physiological evidence exists (e.g. \cite
{Abeles}).

While there has been a fair amount of work in psychology \cite
{Pulvermuller} and neuro-biology to show the existence of CAs, there
has been little computational modelling of CAs \cite {Hetherington,
Huyck, Kaplan}. The computational modeling has either been from a more
abstract level than neurons \cite {Kaplan}, or has been of a very
limited nature \cite {Hetherington, Huyck}.

\section {Quick Activation of CAs}
The theory of CAs states that a CA is quickly activated \cite {Kaplan}.
This shows that slow neurons can quickly recognise an object.

\begin{figure}
\resizebox{\textwidth}{!}{\rotatebox{270}{\includegraphics{snoopy.eps}}}
\centerline{Figure 1.}
\end{figure}

Experiments using simple computational models of CAs shows
this to be the case \cite {Huyck}. This model shows that activation of
10 neurons can lead to activation of the entire CA of 200 neurons in
the next time step. A time step is meant to model 10 ms. of real
neural activity.  This particular experiment is probably not viable
from a neurological standpoint as the size of the CA is very small.
However, it does show how quickly a few neurons can lead to activation
of many more neurons.

A real CA would probably cross several brain areas.  However, it would
only take a few time steps (tens to hundreds of miliseconds) to
activate the entire assembly.  Highly parallel activation compensates
for slow neural speed.  Practically, it will be easy to get large
numbers of neurons activated within the speeds that are needed for
normal real-time performance.

Figure 1 shows the activation curve of a CA.  At time 0, a stimulus is
presented to the network.  Rapidly, a large percentage of the neurons
in the CA become activated and this is associated with recognition.
Stimulus may be removed or remain present.  Some neurons in the CA are
active for quite some time meaning the CA can spread information,
strengthen itself (via the Hebbian learning rule) and remain in
short-term memory.  Note that no neuron remains active for the whole
period; they become active, fatigue, recover and are reactivated by
other neurons in the CA.

\section {Quick Decision on Ambiguous Data}
A human brain must have millions of CAs, but a model network is much
smaller.  A given net may have several CAs in it, but only one or a
few should be active at a given time.  What happens when a net is
presented with ambiguous data?

When ambiguous data is presented to a net, some of the neurons from
two separate CAs will be activated.  These will tend to activate other
neurons in their own CAs. Since the two CAs have never been active
together and reside in the same area of a distance-biased network,
they will tend to inhibit each other.  Thus a competition ensues
between the two ambiguous CAs.  One CA wins, and the data is
recognised as an instance of that type of object.

\begin{figure}
\resizebox{\textwidth}{!}{\includegraphics{ambig.eps}}
\centerline{Figure 2.}
\end{figure}

When the net is presented with ambigous data it does take slightly
longer to recognise the object.  This is due to the extra time needed
for competition.  However, it should only take a few more cycles to
recognise the object, and thus is almost as fast as recognising an
object from unambiguous data.

Figure 2 shows the activation pattern of two CAs. Both get an initial
bust of activating neurons.  Some of these neurons are inhibitory
neurons and inhibit neurons from other CAS this inhibiting the other
CA.  The CA represented by the bold line loses the competition and
returns to its base firing rate.

Of course this is all theory.  I do not know of any model of a CA
which handles ambiguous data \footnote {I am hoping to develop such a
model shortly, and may be able to discuss preliminary results at the
workshop.}.

%\section {The Necker Cube}
%The Necker Cube is clearly an ambiguous object.  However, the earlier
%discussion of ambiguous objects shows that only one object is quickly
%chosen.  The Necker cube ocillates between one interpretation and the
%other.  Why is this?

%Fatigue and persistent ambiguous input.

%One never sees both interpretations at the same time.



\section {Discussion and Conclusion} 
This work applies to two particular areas of interest for the workshop:
processing speed and robustness.  Sections 2 and 3 show how CAs can
quickly recognise objects.  This is due to a large number of simple neurons
working in concert.  These neurons take evidence from the environment, and
using learned connections activate other neurons thus activating the
entire CA and recognising the object.  These neurons work in parallel so
the processing load is distributed.

CAs are robust. Since a CA is spread over so many neurons, the loss of
some of those neurons has a negligible effect on the entire CA.

Assuming CA models can be constructed, how might they be used to
replace the brittle but fast memory presently found in conventional
computers?   CAs can be used to store memories that are based in
reality.  They can solve the symbol grounding problem.  Current
computer memory can store the sentence ``the cat chased the mouse''.
When asked ``what was being chased'' it can respond, but it can not
respond to ``why was he being chased''.

Traditional computer memory is great for number crunching.  CA models 
may be faster than biological CAs but will still be slower than
Von Neumann architectures. CAs however function in context and can 
generate statistically reasonable answers from little data.  They
will function better than Von Neumann architectures in AI tasks like
Natural Language Processing and Decision Support Systems.

The key question for the workshop is {\it What can we learn from
cognitive neuroscience and the brain for building new computational
neural architectures?}  In relation to this paper I would like to
break that question into two questions: \\
1. {\it What can we learn from cognitive neuroscience and the brain for building 
    CAs?}\\
2. {\it What can we learn from CAs for building new computational neural 
    architectures?}

It is simple to say what the brain can tell us about CAs.  The way
each neuron behaves, and the way they are connected in the brain is
the way CAs should behave.  Any CA model will be simplified as
modelling a single neuron is very complex.  Perhaps we can simplify
the model by simply modelling changes in neural activation in discrete
steps and learning by changes of stength.  This is not as robust as
modelling neural activation continuously with ionic transfer between
neurons, and modelling changes of strength by changes in axonal
radius, myelination and geometric properties of the synaptic
cleft. This simplification makes the model run faster which should
enable us to model the large number of neurons needed for CAs.  Still
it is a simplification and, as such, may miss important data.

It is more difficult to say what CAs can tell us about building new computational
neural architectures.  CAs are models for concept storage. They can
solve the symbol grounding problem.  Perhaps the most interesting question is in what
way can CAs work together.

At heart, CAs are a model for data storage.  They have several advantages that
could be transfered to other models.  They remain active for a certain period of
time.  They are a model for both long and short-term memory.  They can be used
to choose between equally likely solutions. It is the reverberating activity that
makes this model different from most existing ANN models.

All neurons are not part of CAs.  Neurons in the eye, send signals
back to the Laterate Geniculate Nucleus and never get activated by
other neurons.  All other neurons may be part of CAs. (The present
data is unclear.)  Those neurons coming from the eye (and other
sensory interfaces) are relatively well understood. Using those
'understood' neurons as inputs to a CA model may enable us to
understand early feature detection.  Thus we can start programming
systems that work on real-world features.

CAs as the basis for combining CAs (cognitive maps).  Activation for a
CA can come from the environment or from other CAs.  They can be used
as models of associative memory, memory of sequences, and hierarchical
memory.  They might be used for rule based systems and other,
traditionally symbolic, activities.  

CAs are not modular though network topologies may be varied.  Despite
our assigning brain areas, the brain is also not modular.  Each area
has billions of connections to and from it.  CAs can act as the basis
of a wide range of architectures for solving a wide variety of
problems in a massively parallel way.

Using CAs as a basis for more complex phenomena puts us on reasonably solid
ground for future exploration of how the brain works at more complex levels.
This in turn leaves us with a better understanding of intelligence.

\begin {thebibliography}{99}

\bibitem {Abeles} Abeles, M., H. Bergman, E. Margalit, and E. Vaadia. (1993) 
Spatiotemporal Firing Patterns in the Fronatal Cortex of Behaving Monkeys.
{\em Journal of Neurophysiology} 70(4):1629-38

\bibitem {Hebb} Hebb, D.O. (1949) The Organization of Behavior. John Wiley and Sons,
New York.

\bibitem {Hetherington} Hetherington, P. A., and M. Shapiro. (1993) 
Simulating Hebb cell assemblies: the necessity for partitioned dendritic trees
and a post-net-pre LTD rule. 
{\em Network: Computation in Neural Systems} 4:135-153

\bibitem {Huyck} Huyck, Christian R. (accepted).  Modelling Cell Assemblies.  
In {\em Proceedings of the International Conference of Artificial Intelligence}
Las Vegas, Nevada

\bibitem {Kaplan} Kaplan, S., M. Sontag, and E. Chown. (1991) 
Tracing recurrent activity in cognitive elements(TRACE): A model
of temporal dynamics in a cell assembly.
{\em Connection Science} 3:179-206

\bibitem {Pulvermuller} Pulvermuller, F. (1999)  Words in the brain's
language.  In {\em Behavioral and Brain Sciences} 22 pp. 253-336.

\end {thebibliography}

\end {document}
