Bayesian Spiking Neurons I: Inference

Denève, Sophie

doi:10.1162/neco.2008.20.1.91

Cited by 252 publications

(263 citation statements)

References 32 publications

Supporting

Mentioning

254

Contrasting

Order By: Relevance

“…This process theory associates the expected probability of a state with the probability of a neuron (or population) firing and the logarithm of this probability with postsynaptic membrane potential. This fits comfortably with theoretical proposals and empirical work on the accumulation of evidence (Kira, Yang, & Shadlen, 2015) and the neuronal encoding of probabilities (Deneve, 2008), while rendering the softmax function a (sigmoid) activation function that converts membrane potentials to firing rates. The postsynaptic depolarization caused by afferent input can now be interpreted in terms of free energy gradients (i.e., state prediction errors) that are linear mixtures of firing rates in other neurons (or populations).…”

Section: Belief Updating and Belief Propagationsupporting

confidence: 81%

Active Inference: A Process Theory

et al. 2017

View full text Add to dashboard Cite

This is the published version of the paper.This version of the publication may differ from the final published version. behavior prescribed by these dynamics has a degree of face validity, providing a formal explanation for reward seeking, context learning, and epistemic foraging. Technically, the fact that a gradient descent appears to be a valid description of neuronal activity means that variational free energy is a Lyapunov function for neuronal dynamics, which therefore conform to Hamilton's principle of least action. Permanent repository link

show abstract

Section: Belief Updating and Belief Propagationsupporting

confidence: 81%

Active Inference: A Process Theory

et al. 2017

View full text Add to dashboard Cite

show abstract

“…This makes the recognition density an approximate conditional density. This corresponds to Bayesian inference on the causes of sensory signals and provides a principled account of perception; i.e., the Bayesian brain (Helmholtz 1860(Helmholtz /1962Barlow 1969;Ballard et al 1983;Mumford 1992;Dayan et al 1995;Rao and Ballard 1998;Lee and Mumford 2003;Knill and Pouget 2004;Kersten et al 2004;Friston and Stephan 2007;Deneve 2008). Finally, it shows that free-energy is an upper bound on surprise because the divergence cannot be less than zero: Optimizing the recognition density makes the free-energy a tight bound on surprise; when the recognition and conditional densities coincide, free-energy is exactly surprise and perception is veridical.…”

Section: Free-energy Action and Perceptionmentioning

confidence: 99%

Action and behavior: a free-energy formulation

et al. 2010

View full text Add to dashboard Cite

We have previously tried to explain perceptual inference and learning under a free-energy principle that pursues Helmholtz's agenda to understand the brain in terms of energy minimization. It is fairly easy to show that making inferences about the causes of sensory data can be cast as the minimization of a free-energy bound on the likelihood of sensory inputs, given an internal model of how they were caused. In this article, we consider what would happen if the data themselves were sampled to minimize this bound. It transpires that the ensuing active sampling or inference is mandated by ergodic arguments based on the very existence of adaptive agents. Furthermore, it accounts for many aspects of motor behavior; from retinal stabilization to goal-seeking. In particular, it suggests that motor control can be understood as fulfilling prior expectations about proprioceptiveThe free-energy principle is an attempt to explain the structure and function of the brain, starting from the fact that we exist: This fact places constraints on our interactions with the world, which have been studied for years in evolutionary biology and systems theory. However, recent advances in statistical physics and machine learning point to a simple scheme that enables biological systems to comply with these constraints. If one looks at the brain as implementing this scheme (minimizing a free-energy bound on disorder), then many aspects of its anatomy and physiology start to make sense. In this article, we show that free-energy can be reduced by selectively sampling sensory inputs. This leads to adaptive responses and provides a new view of how movement control might work in the brain. The main conclusion is that we only need to have expectations about the sensory consequences of moving in order to elicit movement. This means we that can replace the notion of desired movements with expected movements and understand action in terms of perceptual expectations.The Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, 12 Queen Square, London, WC1N 3BG, sensations. This formulation can explain why adaptive behavior emerges in biological agents and suggests a simple alternative to optimal control theory. We illustrate these points using simulations of oculomotor control and then apply to same principles to cued and goal-directed movements. In short, the free-energy formulation may provide an alternative perspective on the motor control that places it in an intimate relationship with perception.

show abstract

“…Much of this research however relies on noisy, stochastic spiking neurons that are characterized by a spike-density, and Bayesian inference is implicitly carried out by large populations of such neurons [9,188,141,183,133,49,17,64,95]. As noted by Deneve [38], coding probabilities with stochastic neurons "has two major drawbacks. First,[...], it adds uncertainty, and therefore noise, to an otherwise deterministic probability computation.…”

Section: Other Snn Research Tracksmentioning

confidence: 99%

“…In [38,39], an alternative approach is developed for binary log-likelihood estimation in an SNN. Such binary log-likelihood estimation in an SNN has some known limitations: It can only perform exact inference in a limited family of generative models, and in a hierarchical model, only the objects highest in the hierarchy truly have a temporal dynamic.…”

Section: Other Snn Research Tracksmentioning

confidence: 99%

Computing with Spiking Neuron Networks

Paugam-Moisy

Bohté

2012

Handbook of Natural Computing

202

114

View full text Add to dashboard Cite

Spiking Neuron Networks (SNNs) are often referred to as the 3 rd generation of neural networks. Highly inspired from natural computing in the brain and recent advances in neurosciences, they derive their strength and interest from an accurate modeling of synaptic interactions between neurons, taking into account the time of spike firing. SNNs overcome the computational power of neural networks made of threshold or sigmoidal units. Based on dynamic event-driven processing, they open up new horizons for developing models with an exponential capacity of memorizing and a strong ability to fast adaptation. Today, the main challenge is to discover efficient learning rules that might take advantage of the specific features of SNNs while keeping the nice properties (general-purpose, easy-to-use, available simulators, etc.) of traditional connectionist models. This chapter relates the history of the "spiking neuron" in Section 1 and summarizes the most currently-in-use models of neurons and synaptic plasticity in Section 2. The computational power of SNNs is addressed in Section 3 and the problem of learning in networks of spiking neurons is tackled in Section 4, with insights into the tracks currently explored for solving it. Finally, Section 5 discusses application domains, implementation issues and proposes several simulation frameworks.

show abstract

Bayesian Spiking Neurons I: Inference

Cited by 252 publications

References 32 publications

Active Inference: A Process Theory

Active Inference: A Process Theory

Action and behavior: a free-energy formulation

Computing with Spiking Neuron Networks

Contact Info

Product

Resources

About