Belief Propagation in Networks of Spiking Neurons

Steimer, Andreas; Maass, Wolfgang; Douglas, Rodney J.

doi:10.1162/neco.2009.08-08-837

Cited by 46 publications

(46 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several recent studies have considered how single neurons or their networks could implement belief propagation [52,55], or of how they could perform probabilistic computations in general [22,23,44,69]. Steimer et al [62] have shown how pools of spiking neurons can be used to implement the Belief-Propagation algorithm on a factor graph. The pools of neurons implement the nodes of a factor graph.…”

Section: Neuromorphic Cognitionmentioning

confidence: 99%

Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition

2009

View full text Add to dashboard Cite

Neuromorphic engineering (NE) is an emerging research field that has been attempting to identify neural types of computational principles, by implementing biophysically realistic models of neural systems in Very Large Scale Integration (VLSI) technology. Remarkable progress has been made recently, and complex artificial neural sensory-motor systems can be built using this technology. Today, however, NE stands before a large conceptual challenge that must be met before there will be significant progress toward an age of genuinely intelligent neuromorphic machines. The challenge is to bridge the gap from reactive systems to ones that are cognitive in quality. In this paper, we describe recent advancements in NE, and present examples of neuromorphic circuits that can be used as tools to address this challenge. Specifically, we show how VLSI networks of spiking neurons with spike-based plasticity mechanisms and soft winner-take-all architectures represent important building blocks useful for implementing artificial neural systems able to exhibit basic cognitive abilities.

show abstract

Section: Neuromorphic Cognitionmentioning

confidence: 99%

Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition

2009

View full text Add to dashboard Cite

show abstract

“…From this point of view, the parameter translation method raises the neural hardware and the computation they are able to achieve to a level of usability that until now was unaccessible by neuromorphic engineers. This ability can be expected to accelerate research on hardware emulation of interesting neuronal computational processes, such as hardware applications of liquid state machines (Maass, Natschläger, & Markram, 2002), and the implementation in spiking neurons of graphical models (Steimer, Maass, & Douglas, 2009). Rutishauser and Douglas (2009) have shown how state-machines can be composed of interconnected networks of winner-take-all (WTA).…”

Section: Discussionmentioning

confidence: 99%

A Systematic Method for Configuring VLSI Networks of Spiking Neurons

et al. 2011

Self Cite

View full text Add to dashboard Cite

An increasing number of research groups are developing custom hybrid analog/digital very large scale integration (VLSI) chips and systems that implement hundreds to thousands of spiking neurons with biophysically realistic dynamics, with the intention of emulating brainlike real-world behavior in hardware and robotic systems rather than simply simulating their performance on general-purpose digital computers. Although the electronic engineering aspects of these emulation systems is proceeding well, progress toward the actual emulation of brainlike tasks is restricted by the lack of suitable high-level configuration methods of the kind that have already been developed over many decades for simulations on general-purpose computers. The key difficulty is that the dynamics of the CMOS electronic analogs are determined by transistor biases that do not map simply to the parameter types and values used in typical abstract mathematical models of neurons and their networks. Here we provide a general method for resolving this difficulty. We describe a parameter mapping technique that permits an automatic configuration of VLSI neural networks so that their electronic emulation conforms to a higher-level neuronal simulation. We show that the neurons configured by our method exhibit spike timing statistics and temporal dynamics that are the same as those observed in the software simulated neurons and, in particular, that the key parameters of recurrent VLSI neural networks (e.g., implementing soft winner-take-all) can be precisely tuned. The proposed method permits a seamless integration between software simulations with hardware emulations and intertranslatability between the parameters of abstract neuronal models and their emulation counterparts. Most important, our method offers a route toward a high-level task configuration language for neuromorphic VLSI systems. An increasing number of research groups are developing custom hybrid analog/digital very large scale integration (VLSI) chips and systems that implement hundreds to thousands of spiking neurons with biophysically realistic dynamics, with the intention of emulating brainlike real-world behavior in hardware and robotic systems rather than simply simulating their performance on general-purpose digital computers. Although the electronic engineering aspects of these emulation systems is proceeding well, progress toward the actual emulation of brainlike tasks is restricted by the lack of suitable high-level configuration methods of the kind that have already been developed over many decades for simulations on general-purpose computers. The key difficulty is that the dynamics of the CMOS electronic analogs are determined by transistor biases that do not map simply to the parameter types and values used in typical abstract mathematical models of neurons and their networks. Here we provide a general method for resolving this difficulty. We describe a parameter mapping technique that permits an automatic configuration of VLSI neural networks so that their electr...

show abstract

“…5) If the graphical models are defined over singly-connected graphs (i.e. trees), then the marginals can be efficiently and exactly computed by belief propagation (BP) algorithms.…”

Section: §1 Introductionmentioning

confidence: 99%

A Family of CCCP Algorithms Which Minimize the TRW Free Energy

Nishiyama

Yuille

2012

New Gener. Comput.

View full text Add to dashboard Cite

We propose a family of convergent double-loop algorithms which minimize the TRW free energy. These algorithms are based on the concave convex procedure (CCCP) so we call them TRW-CCCP. Our formulation includes many free parameters which specify an infinite number of decompositions of the TRW free energy into convex and concave parts. TRW-CCCP is guaranteed to converge to the global minima for any settings of these free parameters, including adaptive settings if they satisfy conditions defined in this paper. We show that the values of these free parameters control the speed of convergence of the the inner and outer loops in TRW-CCCP. We performed experiments on a two-dimensional Ising model observing that TRW-CCCP converges to the global minimum of the TRW free energy and that the convergence rate depends on the parameter settings. We compare with the original message passing algorithm (TRW-BP) by varying the difficulty of the problem (by adjusting the energy function) and the number of iterations in the inner loop of TRW-CCCP. We show that on difficult problems TRW-CCCP converges faster than TRW-BP (in terms of total number of iterations) if few inner loop iterations are used.

show abstract

Belief Propagation in Networks of Spiking Neurons

Cited by 46 publications

References 31 publications

Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition

Artificial Cognitive Systems: From VLSI Networks of Spiking Neurons to Neuromorphic Cognition

A Systematic Method for Configuring VLSI Networks of Spiking Neurons

A Family of CCCP Algorithms Which Minimize the TRW Free Energy

Contact Info

Product

Resources

About