A vlsi recurrent network of integrate-and-fire neurons connected by plastic synapses with long-term memory

Abstract

Abstract-Electronic neuromorphic devices with on-chip, on-line learning should be able to modify quickly the synaptic couplings to acquire information about new patterns to be stored (synaptic plasticity) and, at the same time, preserve this information on very long time scales (synaptic stability). Here, we illustrate the electronic implementation of a simple solution to this stability-plasticity problem, recently proposed and studied in various contexts. It is based on the observation that reducing the analog depth of the synapses to the extreme (bistable synapses) does not necessarily disrupt the performance of the device as an associative memory, provided that 1) the number of neurons is large enough; 2) the transitions between stable synaptic states are stochastic; and 3) learning is slow. The drastic reduction of the analog depth of the synaptic variable also makes this solution appealing from the point of view of electronic implementation and offers a simple methodological alternative to the technological solution based on floating gates. We describe the full custom analog very large-scale integration (VLSI) realization of a small network of integrate-and-fire neurons connected by bistable deterministic plastic synapses which can implement the idea of stochastic learning. In the absence of stimuli, the memory is preserved indefinitely. During the stimulation the synapse undergoes quick temporary changes through the activities of the pre-and postsynaptic neurons; those changes stochastically result in a long-term modification of the synaptic efficacy. The intentionally disordered pattern of connectivity allows the system to generate a randomness suited to drive the stochastic selection mechanism. We check by a suitable stimulation protocol that the stochastic synaptic plasticity produces the expected pattern of potentiation and depression in the electronic network. The proposed implementation requires only 69 83 m 2 for the neuron and 68 47 m 2 for the synapse (using a 0.6 m, three metals, CMOS technology) and, hence, it is particularly suitable for the integration of a large number of plastic synapses on a single chip.Index Terms-Integrate-and-fire neurons, learning systems, neuromorphic aVLSI, synaptic plasticity.

A Time Multiplexing Architecture for Inter-neuron Communications

“…For a fully connected feedforward NN this equates to 9 million synapses. While this is a significant improvement from what is reported elsewhere [13], it will be further enhanced as technology improvements continue [15]. Furthermore, given that the interconnect density will be substantially reduced by the proposed TMA then the real estate given over to the sampling circuitry is expected to be in excess of the 10% estimate.…”

“…Even if we sample at 2*F S to minimise pulse transmission errors, then equation (2) predicts an upper limit for n of half a million. This is an improvement over what is currently achieveable [13]. However, it is clear that the scale of a SNN implemented using the proposed TMA is unlikely to be severely limited by the frequency of the global clock, rather scaleability will be limited by the real estate occupied by circuitry, and the following is an estimate of this limit.…”

Implementing Neural Models in Silicon

“…Hardware implementations offer this possibility. Both digital [90,91] and analogue [58,[92][93][94][95] implementations have been built. Digital implementations using the direct approach are attractive, since we can update the representation of the membrane voltage each timestep.…”

“…Unless one is willing to manually trim V gs for each neuron, this also requires reproducibility of below threshold currents across the chip. Chicca et al [95] used careful layout, with an additional metal layer, but report about 16% variation in leakage current over one chip.…”

“…This form of aVLSI implementation has been used to detect synchrony by taking advantage of the tendency of this implementation of the rule towards making weights go to one of their endpoints [104]. Chicca et al [95] have implemented a bistable excitatory Hebbian synapse. Paired presynaptic spiking input and postsynaptic neural activity results in the synapse being strengthened towards its higher level, but otherwise the synapse decays towards its lower level.…”