NEST for Associative Memory with Winner Take All Nets

Christian Huyck1, Alberto Vergani2

1 Middlesex University, London, UK
2 Aix-Marseille University, Marseille, France

Email: c.huyck@mdx.ac.uk
Using NEST [1] and PyNN [2], an extensible mechanism for representing continuously valued concepts in a network of spiking neurons was developed.  This implemented a continuously valued cell assembly, which had persistent firing to represent short term memory.  This mechanism was embedded in an associative memory [3] associating discrete beverage (coffee and coke) and temperature (hot, cold and warm) concepts with continuously valued temperatures.  Two inputs (e.g. hot and coffee) retrieved the associated concept (e.g. 82⁰-87⁰).  Continuously valued concepts were represented by Winner Take All Networks.  In this case, two dimensional bump attractors with local connectivity were used, but the mechanism should translate to higher dimensional attractors.  The associative memory context led to an issue of a wide range of values being activated, leading to problems with attractor dynamics and multiple streams of values firing instead of a single stream.  An exploration of the attractor dynamics of the system supported the development of a mechanism to resolve the problem; extra topology was added to resolve this problem via merging streams then overcoming the streams repelling each other.  A compensatory Hebbian learning rule is also applied to learn the associations.  This led to results that converge in psychologically realistic time; the system is thus a simple neurocognitive model.  Standard NEST synapse and neuron models are used; Hebbian learning is done between epochs using python, though it can be done as a user defined NEST synapse model, or indeed, as the system uses PyNN, in SpiNNaker.   The system can readily be integrated with the NEAL component architecture to develop more sophisticated agents.

Acknowledgements
This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 720270 (the Human Brain Project) and the UK EPSRC grants EP/D059720 and EP/P00542X/1.  
References

[1] 	M. Gewaltig and M. Diesmann, "NEST (NEural Simulation Tool)," Scholarpedia, 2007. 	
[2] 	A. Davidson, D. Bruderle, J. Eppler, E. Muller, D. Pecevski, L. Perrinet and P. Yqer, "PyNN: a common interface for neuronal network simulators," Frontiers in neuroinformatics, 2008. 	
[3] 	C. Huyck and A. Vergani, "Hot coffee: associative memory with bump attractor cell assemblies of spiking neurons," Journal of Computational Neuroscience.