Embedding Multiple Trajectories in Simulated Recurrent Neural Networks in a Self-Organizing Manner

Abstract

Complex neural dynamics produced by the recurrent architecture of neocortical circuits is critical to the cortex's computational power. However, the synaptic learning rules underlying the creation of stable propagation and reproducible neural trajectories within recurrent networks are not understood. Here, we examined synaptic learning rules with the goal of creating recurrent networks in which evoked activity would: (1) propagate throughout the entire network in response to a brief stimulus while avoiding runaway excitation; (2) exhibit spatially and temporally sparse dynamics; and (3) incorporate multiple neural trajectories, i.e., different input patterns should elicit distinct trajectories. We established that an unsupervised learning rule, termed presynaptic-dependent scaling (PSD), can achieve the proposed network dynamics. To quantify the structure of the trained networks, we developed a recurrence index, which revealed that presynaptic-dependent scaling generated a functionally feedforward network when training with a single stimulus. However, training the network with multiple input patterns established that: (1) multiple non-overlapping stable trajectories can be embedded in the network; and (2) the structure of the network became progressively more complex (recurrent) as the number of training patterns increased. In addition, we determined that PSD and spike-timing-dependent plasticity operating in parallel improved the ability of the network to incorporate multiple and less variable trajectories, but also shortened the duration of the neural trajectory. Together, these results establish one of the first learning rules that can embed multiple trajectories, each of which recruits all neurons, within recurrent neural networks in a self-organizing manner.

Strategies to associate memories by unsupervised learning in neural networks

“…The learning process was based on the work by Liu and Buonomano [5], where a trial, τ, is de¿ned as the network response after a spatial input. All the synaptic modi¿cations are applied after each trial, since the time window of a trial is less than 150 ms, which could match synaptic plasticity time scales.…”

“…Excitatory synapses were modi¿ed according to both homeostatic and STDP rules used in ref. [5], with the same parameter values. Short-term plasticity (STP) was implemented as described in ref.…”

“…The synaptic modi¿cations are applied to excitatory connections, including a homeostatic plasticity and STDP, as described in ref. [5]. The homeostasis rule is used to increase the synaptic weights until the network presents a desired activity and thereafter to maintain a stable activity.…”

“…The main mechanism of synaptic modi¿cation that is responsible for the emergence of associative memory in an unsupervised way is Spike-Timing-Dependent Plasticity (STDP) [2,3], which is a hebbian-like rule. Addionally to the associative memory mechanism, the cells present homeostasis, which has been studied in theoretical works [4,5].…”

Unsupervised learning in neural networks with short range synapses

“…These attempts converged to a more realistic description of the phenomena and has enriched the knowledge of this ¿eld. Versions of model neurons [2] adapted to speci¿c needs and conditions have been proposed along the last ¿fty years, but just rather recently experiments have advanced to allow for a detailed description of Hebbian like synapses [3,4] and on mechanisms to regulate network homeostasis [6,7]. Detailing the connections is also a hard task and it is frequently supposed that they happen involve many neurons.…”