\documentclass{ecai2000}
\usepackage{times}

\ecaisubmission

\begin{document}

\title{Modelling Cell Assemblies}

\author{
Christian R. Huyck \\
Middlesex University\\
c.huyck@mdx.ac.uk\\} 

\maketitle

\bibliographystyle{ecai2000}

\begin {abstract}
People use neurons to store concepts.  Hebb's Cell Assembly (CA)
hypothesis \cite {Hebb} states that concepts, the basis of thought,
are stored in recurrent networks of neurons.  This paper presents a
novel model CANT, which exhibits CA-like behavior.

The CANT model is driven by our understanding of biological neurons and
networks.  It uses activation, uni-directional connections, connection
strength varying by a Hebbian learning rule, distance-biased connectivity,
neural fatigue and neural decay to model biological neural nets.

Two experiments are explained which learn patterns in an unsupervised
manner.  Statistical evaluations of the network are used to show that
CAs are present.  The networks are compared to Hetherington's network
\cite {Hetherington} the best prior model of CAs, and the CANT
model is shown to be superior.
\end{abstract}

\section {Introduction and Background} 
Neural Networks are very popular computational models that are
generally said to be inspired by human neural functioning.  However,
'neurons' in most neural nets function quite differently from human
neurons.

An average human neuron fires for a brief time, and then fatigues.  If
such a neuron were to represent a concept, we could only think of that
concept for a brief time. Since we can think of a given concept for
at least several seconds, a single neuron cannot represent an entire concept.
A concept must be represented by a collection of neurons.  How might
such a concept be represented?

D. O. Hebb proposed a solution to this problem \cite {Hebb}, a
reverberating circuit of cells. Hebb called this reverberating circuit
a {\em Cell Assembly} (CA). Some physiological evidence exists; for
instance, Abeles \cite {Abeles} has experimented with electrical
probes in the brains of monkeys.  There are 10 probes in a small
($\leq$ 5 cm) area of the brain.  The electrodes show repeating
patterns of activation when the monkey is sensing or performing a
specific event.

While there has been a fair amount of work in psychology \cite
{Pulvermuller} and neuro-biology to show the existence of CAs, there
has been little computational modelling of CAs \cite {Hetherington}
\cite {Kaplan}. The computational modelling has either been from a 
more abstract level than neurons \cite {Kaplan}, or has been of 
a limited nature \cite {Hetherington}.

This paper describes a novel architecture, CANT (Connections,
Associations and Network Technology).  Experiments using CANT are
described and show how CANT can represent CAs.

\section {Basic Neural Model} 
There is a significant amount of knowledge about how biological
neurons function, and how the neural systems they are embedded in
function.  However, this knowledge is far from complete.  Due to the
complexity of biological neural systems, it is difficult to discover
how neurons and neural systems actually enable us to think.  The goal
of this research is to develop models that are based on our
understanding of biological neural systems and adhere to the
constraints of those systems.

This system is thus a model of biological neural systems.  As it is a model,
some simplifying assumptions are made.  Hopefully, these assumptions
will allow the system to keep the essential detail of the biological neural
systems.

The model is based on neurons. These neurons have axons, which are
unidirectional connections to other neurons.  Neurons become active
and when they are activated, they send activation down their axons to
adjacent neurons.  These properties are explained further below, but
are typical to most artificial neural systems.

Three other properties are generally not used in artificial neural networks
but are parts of biological neural systems: Distance Biased Connectivity,
Neural Decay and Neural Fatigue.

\subsection {Connections and Connection Strength}
A CANT neuron has connections to other neurons, which are similar to
connections inside biological neural systems.  Connections are
unidirectional.  Like most neural net simulations, the connection
strength may vary based on a learning rule.  The connection may have
positive or negative strength.  Continuous activation is simulated by
time steps.

The average biological neuron is activated by about 1000 other
neurons, and in turn activates about 1000 other neurons \cite
{Lippmann}.  Each biological neuron has several axons, which have feet
to send activation to other neurons.  The CANT model simply has
neurons and connections.  In the experiments described in this paper,
there are fewer connections than a biological net; due to the small
number of neurons used in the experiments.  In future experiments with
larger networks, more neurons and connections will be used.

\subsection {Activation}
When the neuron crosses a threshold, it sends activation down each of its
axons. The activation of a given neuron i at time t is:
$$h_{i_t} = {{h_{i_{t-1}}}\over d} + \sum_{j \in V_i} w_{ij} s_j $$
\centerline{\bf{Equation 1.}} \\ The current activation is the activation
from the last time step divided by a decay factor {\it d} plus the new
activation coming in.  This new activation is the sum of the active
inputs $s_j$ of all neurons $ j \in V_i$, $V_i$ being the set of all
neurons that are connected to {\it i}, weighted by the value of the
connection from neuron {\it j} to neuron {\it i}.

%removed for space
%\begin{picture}(300,130)(0,0)
%\thicklines
%\put(160,70){\circle{40}}
%input axons
%\put(120,70){\vector(1,0){20}}
%\put(130,40){\vector(1,1){15}}
%\put(130,100){\vector(1,-1){15}}
%output axons
%\put(181,70){\vector(1,0){20}}
%\put(181,72){\vector(1,1){15}}
%\put(181,68){\vector(1,-1){15}}
%\put(140,10){\bf{Figure 1.}}
%\end{picture}

There may also be external activation.  Theoretically this comes from
the environment, but in these experiments it comes from neurons being
directly activated. The first experiment allows neurons to
spontaneously activate when they have been inactive for a long time.

A neuron is active if and only if it has enough activation to surpass
the activation threshold.  Each neuron has the same activation
threshold as all other neurons.  So if the activation threshold is 5
and a given neuron has an activation of 4, it will not fire and thus
will not propagate activation.  Activation of an inactive neuron does
decay but an inactive neuron does not fatigue.


\subsection {Distance-Biased Connectivity}
Biological neural systems are connected in a distance-biased way.
%\cite {undone}.  
Each neuron is not connected to every other neuron; in the
human brain, with 500 billion neurons, this would require each neuron
to have 500 billion connections, and as we have seen the average
neuron has approximately 1000 connections.  If two neurons are closer
together they are more likely to be connected. The CANT model adheres
to this distance-bias connectivity.

\subsection {Neural Decay}
At each step, the activation from the last step decays.  Of course,
new activation may lead to a net gain in activation.  Decay is a
constant and applies to active and inactive neurons. In experiment 2,
active neurons lose all activation thus decay is irrelevant for them.

\subsection {Neural Fatigue}
The neuron fatigues.  When a biological neuron is active for a long
time it will fatigue and this will make it less likely to remain
active.  This is modelled by a fatigue factor, which increases the activation
threshold.  The threshold is increased by  ${f_c t_a} $. 
$f_c$ is the fatigue constant and
$t_a$ is the time that the neuron has been active.  The longer that the
neuron is active the larger the threshold becomes, and thus the less likely
it is to remain active.

When a neuron becomes inactive, fatigue is reduced.  Fatigue is 
$t_a - {t_r R_c} $.  Fatigue is the time active minus the time
recovering multiplied by the recovery constant.  The higher the
constant, the faster a neuron recovers from fatigue.  

%remove for space
%In the models described in this paper, a neuron tends to stay active
%for at most 10 cycles; this varies with decay and fatigue.  Even with
%one decay and fatigue setting this does of course vary.  A neuron may
%remain active for as little as one cycle, and if given enough
%activation, may remain active indefinitely.  In the experiments
%described in this paper, the longest a neuron has remained active is
%13 cycles.

\subsection {Learning}
Learning is based on a Hebbian model.  That is, learning occurs locally.
If neuron A sends activation via an axon to neuron B, and both A and
B are active, then the strength of the axon is increased; this is the
Hebbian learning rule.  If neuron B is not active then the strength of
the axon is reduced; this is the anti-Hebbian learning rule.

Increasing (or decreasing) strength is all that is called for by these
rules and this leaves a great deal of room for interpretation of how
much strength is added.  It is not clear what precise strengths are added
from the biological data, but some limits are obvious.  Firstly, an axon
cannot gain unlimited strength; it should have some limit. Secondly, it
is plausible that the strength of all of the axons of a neuron may have
some limit.  The experiments below use two different learning rules.

\section {Experiments}
The basic CANT model is set, but within the model there is a vast
range of possible networks.  This section explains two networks and
how they exhibit CA-like behavior.  First the topology that both sets
of nets share is described.  Second, the mechanism used to measure CA
presence is explained.  Finally, the experiments themselves are
described.


\subsection {Topology}
The network has a Toroidal Topology.  It is a square matrix of
neurons.  Due to distance-biasing, each neuron is more likely to be
connected to its neigboring neurons.  The toroidal topology implies
that neurons on the left side of the matrix are adjacent to neurons on
the right side of the matrix, and neurons on the top are adjacent to
those on the bottom.

This is not an accurate model of the brains topology, but it is a
reasonable starting point.  We are not modelling the entire brain or
even brain areas with hundreds of neurons.


\subsection {Measuring CAs}
How does one measure for the presence of CAs?
It is hoped that CAs are {\it unique}, {\it persist}, and are {\it
reliably activated}.  Statistical measurements of the entire network
at certain stages can show these properties.

Uniqueness can be measured by comparing activation patterns of
different CAs. When an A pattern is presented to the network it should
lead to the activation of different neurons than if a B pattern is
activated \footnote {While there is nothing theoretically to stop a
neuron from participating in two CAs, in the current experiment a
neuron should not participate in two CAs.  Each input pattern is
entirely separate.  It's like comparing As to Os; they have no
overlapping features.  A different pair of input patterns, say Es and
Fs, might contain overlapping neurons in the CA.}.

Persistence can be measured by presenting a stimulus to a network, and
then comparing the activation patterns in early cycles and in later
cycles.  If the two are correlated, then the CA persists.

A CA is reliably activated if different inputs of a similar pattern
lead to similar neurons being activated.  For instance, one A pattern
can be presented and the state of the network can be measured after a
given number of cycles.  When a second (but different) A pattern is
presented, it should lead to the activation of a similar set of
neurons.

The statistical measurement used in this evaluation is Pearson's
product moment correlation coefficient.  The measurement that is used
for the network is whether a given neuron is active or not.  The
Pearson's correlation is used because it ignores the inactive neurons,
and in all cases, most of the neurons are inactive.

\subsection {Experiment 1: Learning CAs in a Network } 
%undone Need to say something about random activation see paragraphs
% 4 and 6 of this section

%Model 9
The goal is to have one CA activated when one type of pattern is
presented and another activated when a second type of pattern is
presented.  Below it is shown that the network exhibits this behavior.

In a given run, the network is presented with a pattern for 10 cycles.
The network is allowed to run for 1000 cycles in a run.  Since neurons
are activated, the connection strengths will change using the learning
rule.  Consequently, the network will differ (in connection strengths)
from run to run.  CAs are formed by connection strengths changing so
that the appropriate a reverberating circuit is activated when a
pattern is presented.

This system has 900 neurons.  Each pattern had 10 primary activation
points and 10 secondary activation points.  The points of pattern A
were between 0 and 100 while those for pattern B were between 500 and
600.  At each training run, 6 primary and 4 secondary neurons from
either A or B were activated.  The type of activation (A or B) was
randomly selected, as were the neurons in any given pattern.

Neurons may also spontaneously activate.  If a neuron has not fired in
many (hundreds of) cycles, it may spontaneously activate.  This mimics
auto-activation in the brain. An example of auto-activation is when a
person hallucinates while in a sensory isolation tank.  

Learning includes a compensatory learning mechanism.  If an axon is
strengthened then the overall axonal strength of the neuron is
increased.  If this strength is above a threshold, then the strength
of all of the axons is reduced to keep the overall axonal strength
below the threshold.

This system is similar to that described in \cite {Hetherington}.
Hetherington's system had 360 neurons and learned 4 patterns, but his
patterns were 5 neurons and all were always turned on.  Additionally,
Hetherington did not use random activation.  Consequently, it could
not recruit neurons into the CA, which had not already been activated.
Hetherington's system will be considered in the discussion section.

Following Hetherington, it is hoped that CAs are {\it unique}, {\it
persist}, and are {\it reliably activated}.  Statistical measurements
of the entire network at certain stages can show these properties.


Table 1. shows different correlations based on the number of training
runs.  These correlations show the development of CAs in the net.

The columns represent the number of training runs.  A training run
consists of a presentation of the stimuli followed by 1000 cycles of
activation propagation.  Table 1 shows the state of the initial net
(0 Runs), the net after 200 training runs, and after 400 runs. In each
case the system was presented with 4 A stimuli and 4 B stimuli, in
different runs.  The table represents the average results.

\begin {tabbing} 
\hspace {1.0 in} \= \hspace {1 in} \= \hspace {.75 in} \= \hspace {.75 in} \= \\ 
\> 0 Runs \> 200 Runs \> 400 Runs  \\
Max. Neurons\> \hspace{0.1 in} 10\> \hspace{0.1 in} 30\> \hspace{0.1 in} 31  \\
A-A Corr.\> \hspace{0.1 in} -.0074\> \hspace{0.1 in} .7585\> \hspace{0.1 in} .7442 \\
B-B Corr.\> \hspace{0.1 in} -.0067\> \hspace{0.1 in} .6456\> \hspace{0.1 in} .8077 \\
A Self Corr.\> \hspace{0.1 in} -.0068\> \hspace{0.1 in} .1649\> \hspace{0.1 in} .2110 \\
B Self Corr.\> \hspace{0.1 in} -.0057\> \hspace{0.1 in} .2175\> \hspace{0.1 in} .2332  \\
A-B Corr.\> \hspace{0.1 in} -.0073\> \hspace{0.1 in} -.0068\> \hspace{0.1 in} -.0351  \\
\centerline{\bf{Table 1}: Network Correlations for Experiment 1}
\end {tabbing}

{\it Max. Neurons} represents the maximum number of neurons that are
activated in any cycle of a run.  Since the cell of the table
represents eight runs, this number represents the average maximum
number.

{\it A-A Corr.} represents the correlation of active neurons between
different runs of type A.  The measurement is taken at cycle 15. {\it
B-B Corr.}  represents the correlation between B runs. Since four runs
occurred, the correlation is the average of the six comparisons.  These
measurements show reliable activation.  If different but similar
patterns are activated, they should send activation to the same neurons,
especially in the initial phase  of CA activation.  The higher this number
the more reliable the CA.

{\it A Self Corr.} shows the correlation between the activation states
of one net in a given run when there is an A stimulus.  The
correlation is between the net after cycle 15 and after cycle 30.
{\it B Self Corr.}  is the version when there is a B stimulus. Since
there are four patterns of these types, this number represents the
average of four runs.  This number is an indicator of persistence
\footnote {It really only shows similarity of activation at two
different parts of a given run.  Since neurons fatigue, the same CA
may be active but different neurons are active.  A better measurement
might be to compare several states (eg. after cycle 15, 30, 45 and
100).  Another measurement might be to see the reverberating activity
of a particular neuron.  Do the input neurons become active again
after fatiguing?  This would parallel \cite{Abeles}. See the
discussion section for further elaboration.}. Similar neurons are
active early in the run and later in the run.  To some extent, the
higher the number, the more persistent the CA.

Finally, {\it A-B Corr.} represents the correlation between runs of
different types.  Again, the state of the net was recorded after cycle
15, and again the average correlation is shown.  This number shows
uniqueness.  If two network activation patterns derived from different
types of inputs are compared, they should not have similar neurons
activated.  The lower this number, the more unique the CAs are.

In a given run, the neurons that are part of the initial pattern (both
primary and secondary) will be externally activated.  Since neurons in
the pattern are both active, learning will strengthen the connection
between them, and they will tend to remain active longer.  As the CA
is formed, those neurons that are in the initial pattern will remain
active in a given run even after the pattern is no longer presented.

Some neurons are part of the general pattern, but not part of the
pattern that is presented in a given run.  When the CA is formed,
those neurons will also tend to become active, because connections to
those neurons in the particular pattern have been strengthened via
learning.

%consider could do better here.
Neurons that are not part of either pattern will be recruited to the
CAs, thus the two CAs are in competition for neurons.  For instance, a
neuron X that is not part of either pattern is being recruited by CA A
will have connections to it from neurons in CA A.  These connections
will be strengthened by the learning rule when pattern A is activated
and X is activated either randomly or via existing connections to A.
Connections from X to neurons in CA A will also be strengthened.
Connections from X to neurons in CA B will have their strengths
reduced and might become inhibitory.

Thus when CA A has evidence that supports it, it will become active,
and it will suppress CA B.  As CA A recruits more neurons it will be
able to suppress CA B more readily, and will thus be able to recruit
more neurons.  Similarly CA B will be trying to recruit neurons from CA
A.

\subsection {Experiment 2:  A Better Neural Model}
%model 11

There are some problems with the model presented in the first
experiment.  Firstly, a given neuron can have both excitatory and
inhibitory axons leaving it; biological neurons have either inhibitory
or excitatory axons but not both.  Secondly, neurons can remain active
even after they fire; biological neurons lose their activation when
they fire.


In experiment 1, connection strength of an axon could vary from
negative to positive and back, based on the learning rule and the
activations of the neurons it connected.  This is not biologically
viable \cite {Vander}.  Biological neurons have either inhibitory or
excitatory axons but not both, the strength of the connection can
vary, but it stays positive or negative.  20 percent of neurons were
inhibitory.

Experiment 1 also had problems with neurons maintaining their
activation after they have fired.  Biologically, this is not
plausible \cite {Vander}.  The leaky integrator model is better.  In
experiment 1, a neuron decayed a certain percentage whether it fired
or not.  In this experiment, the neuron loses all activation if it
fires, but decays if it does not fire.  Consequently, a neuron that
has activation, but not enough activation to surpass the threshold
will activate more easily in the subsequent cycle.

The learning rule in this experiment is: 
$(log_{10}(S+2) - log_{10} (S + 1))*(10^{-S})*R$. 
Where S is Connection Strength and R is the learning rate constant.  
This has the property of growing rapidly
when S is small, but growing very slowly, without a limit, when it
is large. The forgetting rule is a pre-not-post rule.  It is:
$(log_{10} (S + 1))*R$.  This has the property of reducing strength
more when S is large and less when it is small.

The network of experiment 2 used 400 neurons instead of the 900
neurons of experiment 1.  In general, a larger net should support
longer activation curves and more sophisticated patterns.

Experiment 2 also used different types of activation patterns. Each
series of runs used two patterns to form two CAs.  The patterns were
exhaustive.  That is, each neuron was a member of one of the two
patterns.  Since each neuron was already a member of a pattern, random
activation was not needed.

Several types of patterns were explored.  The most basic type of
pattern divided the network in half.  Pattern A consisted of the
neurons 0-199 and pattern B the neurons 200-399.  These patterns were
entirely local.

A range of patterns was used from the above local pattern, up to and
including fully interleaved patterns.  Fully interleaved patterns were
pattern A having all of the even neurons, and pattern B having all of
the odd neurons.  Intermediate patterns combined these; for example
pattern A consisted of neurons 0-99 and the even neurons from 100-299,
pattern B consisted of neurons 300-399 and the odd neurons from
100-299.

\begin {tabbing} 
\hspace {1.0 in} \= \hspace {1 in} \= \hspace {.75 in} \= \hspace {.75 in} \= \\ 
\> Local \> Half \> Interleaved  \\
Number Runs\> \hspace{0.1 in} 300\> \hspace{0.1 in} 350\> \hspace{0.1 in} 1500  \\
A-A Corr.\> \hspace{0.1 in} 1\> \hspace{0.1 in} .9286\> \hspace{0.1 in} .9900 \\
B-B Corr.\> \hspace{0.1 in} 1\> \hspace{0.1 in} .9492\> \hspace{0.1 in} .9917 \\
A Self Corr.\> \hspace{0.1 in} .9970\> \hspace{0.1 in} .8821\> \hspace{0.1 in} .8621\\
B Self Corr.\> \hspace{0.1 in} .9734\> \hspace{0.1 in} .9562\> \hspace{0.1 in} .9386\\
A-B Corr.\> \hspace{0.1 in} -1\> \hspace{0.1 in} -.4973\> \hspace{0.1 in} -.9826  \\
\centerline{\bf{Table 2}: Network Correlations for Experiment 2}
\end {tabbing}


The results of this experiment show that this model is capable of
learning CAs over the full range of exhaustive patterns. Local
patterns, interleaved patterns, and patterns which combined local
groups and interleaved groups all formed CAs.  These patterns
are unique, persist and are reliably activated.

It takes longer to learn interleaved patterns, but they are
eventually learned.  This is because a distance-biased network
improves localized CAs.  Distance-biasing acts as an attractor and
when a localized pattern is placed into that attractor it is
easily learned. Patterns that fight the attractor can still
be learned but take longer, and are less successful.

The CANT model surpasses Hetherington's model.  Comparing CANT using
localized patterns to Hetherington's best model using partitioned
dendretic trees, CANT is almost as reliable (.96 to .97)
\footnote {Here actual real valued activation values are used as
opposed to boolean active or inactive values.  This is to even the
comparison with Hetherington's model.}  , is more unique (-.8487 to
-.13), and is more persistent (.9969 to .95).  It seems that dendretic
partitioning is unnecessary, for this simple type of model, as
distance-biasing gives the same or better results.

\section {Discussion} 
Four issues are particularly important.  The first is comparing 
the quality of different CAs, and a particularly important 
part of this comparison is reverberating activity.  Second
is the dynamics of a CA over different runs and how it gains or
loses neurons. Thirdly, learning mechanisms must be discussed.
Finally, the patterns that are recognized by these systems
are important.

Though the statistics from the second experiment are better than the
statistics from the first, the CAs from the first are in some sense
better.  This is because CAs from the first system actually
reverberate.  CAs from the second system are quite like CAs from
Hetherington's system.  Consequently, CAs from the first system are
better than those from Hetherington's system.  Of course, the second
system is superior to the first system as it is a better model of
biological neurons.

What is needed in a CA is reverberating neural activity.  A neuron
becomes active for several cycles, it then becomes inactive, then it
becomes active again.  Ideally, this reactivation cycle will happen
several times.  This reactivation cycle is needed to maintain the
activation of a concept.  It is rare for a given neuron to remain
active for more than a second, but concepts should remain active for
longer than that.  Consequently, a new set of neurons will need to
become active, which in turn will deactivate.  This is when the initial
neurons will need to be reactivated.

Unfortunately, the persistence measurement does not measure
reverberation.  Instead it measures neurons having similar activation
at different times.  Persistence numbers like .99 show that the same
neurons are active at different times though the overall activation
may be lower.  Due to fatigue, CAs from the second system and from
Hetherington's system are still active though at a lower rate.  They
have never become inactive. CAs from the first system have become
inactive but have been reactivated.  Thus the entire CA can maintain
activation for longer.  Smaller persistence numbers (like .23 from the
first experiment) show that similar neurons are active at different
stages.  A better measurement for reactivation and persistence is
needed.

Over a period of runs, new neurons are added or recruited to CAs, and
others may leave the CA.  This recruitment can lead to the problem of
overgeneralization.  A CA can recruit another CA and thus the CAs
become indistinguishable.  This occurs more easily in interleaved
patterns, as the CAs have more connections to each other.  CA
formation can be seen as a battle between CAs to recruit and maintain
neurons.  This happens with random activation and without, but random
activation makes the problem more difficult.  What is needed is a
mechanism of building up CA boundaries, and then slowly battling over
those boundaries.  At least in these exclusive pattern systems, this
should be done without breaking down the boundaries. Gradual learning
of strengths seems to build up boundaries with minimal breaking down
of the boundaries.  Perhaps inhibition can also play an important
part here.

Learning rules must be local.  However, within this restriction there
is a large range of possibilities.  It appears that the best rules
strengthen more when the axons are weak and less when they are strong.
This enables the CA to develop, but not to become too powerful and 
recruit neighboring CAs.

In these two experiments, we have learned two types of patterns.  The
first experiment learned local incomplete patterns, and the second
used exhaustive patterns.  There are many other types of patterns that
need to be learned if this model is to account for all patterns that
biological systems learn.  Still, learning the full range of
exhaustive patterns is a good start and shows that the model is
somewhat robust.

\section {Conclusion and Future Work } 

This paper describes the CANT model and two instances of it.  The
experiments differ by the patterns they learn, the learning rule they
use, and the manner in which neurons decay. Both experiments exhibit
CAs.  The second experiment surpasses the best known model for CAs,
Hetherington's, on statistical measurements.

Since CAs have been formed, this model is a good one on which to explore
issues about CAs. Further variation on the model could lead to a better
model.

Can a better version of the model be found?  Learning could be
expanded by varying the strength of inhibitory axons.  Additionally,
it seems that fatigue could play a part in learning.  Learning 
could occur in a longer time scale than activation.

The topology could also be varied to include more biologically plausible
networks.  Larger networks could be used and networks with more axons
per neuron.  This topology must effect the model.

The experiments show patterns, which are distinct.  Clearly concepts have
overlapping features and are not distinct.  The model should be able to
learn patterns, which have overlapping features.

The persistence measurement is weak.  We need to find a better measurement.
Statistical techniques should exist for measuring reverberation.

CAs should have two very useful long term properties.  Firstly, they should
form a new type of neural model and thus may be used for AI problems. Secondly,
and more importantly, CAs are a model of human neural processing.  Consequently,
they should be able to elucidate human neural and psychological processes. These
are the long-term goals of this research.
\bibliography {hebb} 

\end {document}

