STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning

Abstract

In order to cross a street without being run over, we need to be able to extract very fast hidden causes of dynamically changing multi-modal sensory stimuli, and to predict their future evolution. We show here that a generic cortical microcircuit motif, pyramidal cells with lateral excitation and inhibition, provides the basis for this difficult but all-important information processing capability. This capability emerges in the presence of noise automatically through effects of STDP on connections between pyramidal cells in Winner-Take-All circuits with lateral excitation. In fact, one can show that these motifs endow cortical microcircuits with functional properties of a hidden Markov model, a generic model for solving such tasks through probabilistic inference. Whereas in engineering applications this model is adapted to specific tasks through offline learning, we show here that a major portion of the functionality of hidden Markov models arises already from online applications of STDP, without any supervision or rewards. We demonstrate the emergent computing capabilities of the model through several computer simulations. The full power of hidden Markov model learning can be attained through reward-gated STDP. This is due to the fact that these mechanisms enable a rejection sampling approximation to theoretically optimal learning. We investigate the possible performance gain that can be achieved with this more accurate learning method for an artificial grammar task.

A Minimal Spiking Neural Network to Rapidly Train and Classify Handwritten Digits in Binary and 10-Digit Tasks

“…Neural networks that use biologically plausible neurons and learning mechanisms have become the focus of a number of recent pattern recognition studies [1,2,3]. Spiking neurons and adaptive synapses between neurons contribute to a new approach in cognition, decision making, and learning [4][5][6][7][8].…”

TrueHappiness: Neuromorphic emotion recognition on TrueNorth

“…While this is probably acceptable for many practically relevant systems, many other systems might need to adjust to a specific user or a new environment. In such cases it would be necessary to use neuromorphic hardware that brings efficient online learning [5], [21], [22] in combination with online learning algorithms [23], [24], [25], [26], [27]. However, so far it remains a challenge to achieve performances comparable to conversion methods using such online learning approaches.…”

Local structure helps learning optimized automata in recurrent neural networks

“…It will be interesting to investigate the effects of local structure in non-deterministic finite automata, as in this case algorithmic minimization of a given automaton is much more expensive than in the deterministic case [1]. Thus, similar state space optimization techniques might prove to be advantageous for neural models of probabilistic decision making [2], [3]. Maze task, in which a virtual agent has to find a target position, using visual cues.…”

“…FSMs can model many aspects of high level deterministic behavior, such as production of movement sequences, navigation, state-dependent decision making, log ical reasoning, or understanding and production of language. Although many neural processes can be better modeled by probabilistic graphical models, taking into account the inherent environmental and neural stochasticity [2], [3], almost deter ministic sequences of neural activation have been observed in brains of various species and during diverse activities. Examples include synfire chains [4], sequences during song production in birds [5], or location-dependent patterns during navigation in rats [6].…”