One-shot learning with spiking neural networks

Abstract

Understanding how one-shot learning can be accomplished through synaptic plasticity in neural networks of the brain is a major open problem. We propose that approximations to BPTT in recurrent networks of spiking neurons (RSNNs) such as e-prop cannot achieve this because their local synaptic plasticity is gated by learning signals that are rather ad hoc from a biological perspective: Random projections of instantaneously arising losses at the network outputs, analogously as in Broadcast Alignment for feedforward networks. In contrast, synaptic plasticity is gated in the brain by learning signals such as dopamine, which are emitted by specialized brain areas, e.g. VTA. These brain areas have arguably been optimized by evolution to gate synaptic plasticity in such a way that fast learning of survivalrelevant tasks is enabled. We found that a corresponding model architecture, where learning signals are emitted by a separate RSNN that is optimized to facilitate fast learning, enables one-shot learning via local synaptic plasticity in RSNNs for large families of learning tasks. The same learning approach also supports fast spike-based learning of posterior probabilities of potential input sources, thereby providing a new basis for probabilistic reasoning in RSNNs. Our new learning approach also solves an open problem in neuromorphic engineering, where onchip one-shot learning capability is highly desirable for spike-based neuromorphic devices, but could so far not be achieved. Our method can easily be mapped into neuromorphic hardware, and thereby solves this problem.Preprint. Under review.

2022 roadmap on neuromorphic computing and engineering

“…Option 3 is used by the MAML approach of [336], where only very few updates of synaptic weights via BPTT are required in the inner loop of L2L. It also occurs in [325] in conjunction with option 4, see figure 24 for an illustration.…”

Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity

“…In this direction, short-term synaptic plasticity [8], [9] and neuronal adaptation [7] have been exploited. One-shot learning of SNNs has been studied in [27]. Instead of Hebbian plasticity, this model relied on a more elaborate three-factor learning rule.…”

“…While standard deep learning approaches need large numbers of training examples during training, humans can learn new concepts based on a single exposure (one-shot learning) [24]. A large number of few-shot learning approaches have been proposed using artificial neural networks [25], [26], but biologically plausible SNN models are very rare [27]. We wondered whether Hebbian plasticity could endow SNNs with one-shot learning capabilities.…”

Fast On-Device Adaptation for Spiking Neural Networks via Online-Within-Online Meta-Learning

“…We first consider the family of omniglot 2-way classification tasks [36], which is often used to test the within-task generalization capabilities of meta-learners [6], [7], [11], [29]. We We train a fully connected GLM-based SNN with 4 hidden neurons and 2 visible neurons with additional lateral connections between the hidden neurons and feedback connections from the visible neurons to the hidden neurons.…”