Free Energy, Value, and Attractors

Friston, Karl J.; Ao, Ping

doi:10.1155/2012/937860

Cited by 109 publications

(139 citation statements)

References 90 publications

Supporting

Mentioning

129

Contrasting

Order By: Relevance

“…Furthermore, free utility depends on a cost function, while free energy does not. This is because the free-energy principle is based on the invariant or ergodic solution P(o|m) to the Kolmogorov forward equation, which specifies the value of an observed state V (o|m) = ln P(o|m) directly, without reference to path integrals or cost (Friston and Ao 2012). Conversely, free utility is based on the Kolmogorov backward equation, which can only be solved given terminal costs.…”

Section: Discussionmentioning

confidence: 99%

“…This formulation of behaviour is based on ergodic arguments about the nature of self organising systems (Ashby 1947)-for a fuller discussion please see Friston and Ao (2012) and Friston (2010) for their neurobiological implications. These arguments suggest that the long term average of variational free energy upper bounds the (Shannon) entropy of observations over time; which implies that action must minimise variational free energy to resist the dispersion of its states by random fluctuations (Evans 2003).…”

Section: The Free-energy Principle and Active Inferencementioning

confidence: 99%

“…A simple example here would be the context and time sensitive nature of motivational and physiological drives (Berridge 2004) that render the same states differentially rewarding, depending upon when they were last visited (for example, eating or drinking). In continuous time formulations of active inference, this dependency is usually dealt with in the context of dynamical systems theory, where desired states become attracting sets, which can either be fixed point attractors or more complicated trajectories on attractor manifolds (Friston and Ao 2012). In this setting, the representation of a trajectory over successive states (s 0 , s 1 , s 2 , .…”

Section: Simulation Setupmentioning

confidence: 99%

“…Crucially, both action and perception (estimating the hidden causes of sensory input) are trying to minimise the same thing-roughly speaking, prediction error or the surprise associated with sensations. This is active inference in its simplest form and has been considered in the context of reinforcement learning ), action selection ) optimal motor control (Friston 2011) and dynamical systems theory (Friston and Ao 2012). However, all of these treatments consider behaviour in continuous time, in contrast to the discrete state space and time formulations that predominate in the literature on optimal decision problems.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Active inference and agency: optimal control without cost functions

2012

Self Cite

View full text Add to dashboard Cite

This paper describes a variational free-energy formulation of (partially observable) Markov decision problems in decision making under uncertainty. We show that optimal control can be cast as active inference. In active inference, both action and posterior beliefs about hidden states minimise a free energy bound on the negative log-likelihood of observed states, under a generative model. In this setting, reward or cost functions are absorbed into prior beliefs about state transitions and terminal states. Effectively, this converts optimal control into a pure inference problem, enabling the application of standard Bayesian filtering techniques. We then consider optimal trajectories that rest on posterior beliefs about hidden states in the future. Crucially, this entails modelling control as a hidden state that endows the generative model with a representation of agency. This leads to a distinction between models with and without inference on hidden control states; namely, agency-free and agency-based models, respectively.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: The Free-energy Principle and Active Inferencementioning

confidence: 99%

Section: Simulation Setupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Active inference and agency: optimal control without cost functions

2012

Self Cite

View full text Add to dashboard Cite

show abstract

“…The barrier heights are shown to be associated with the escape time from one state to another. Therefore, the topography of the landscape through barrier height can provide a good measurement of global stability and function (12,(14)(15)(16)(17). Such probability landscape can be constructed through solving the corresponding probabilistic equation.…”

Section: Significancementioning

confidence: 99%

Nonequilibrium landscape theory of neural networks

Yan

Zhao

et al. 2013

Proc. Natl. Acad. Sci. U.S.A.

144

View full text Add to dashboard Cite

The brain map project aims to map out the neuron connections of the human brain. Even with all of the wirings mapped out, the global and physical understandings of the function and behavior are still challenging. Hopfield quantified the learning and memory process of symmetrically connected neural networks globally through equilibrium energy. The energy basins of attractions represent memories, and the memory retrieval dynamics is determined by the energy gradient. However, the realistic neural networks are asymmetrically connected, and oscillations cannot emerge from symmetric neural networks. Here, we developed a nonequilibrium landscape-flux theory for realistic asymmetrically connected neural networks. We uncovered the underlying potential landscape and the associated Lyapunov function for quantifying the global stability and function. We found the dynamics and oscillations in human brains responsible for cognitive processes and physiological rhythm regulations are determined not only by the landscape gradient but also by the flux. We found that the flux is closely related to the degrees of the asymmetric connections in neural networks and is the origin of the neural oscillations. The neural oscillation landscape shows a closed-ring attractor topology. The landscape gradient attracts the network down to the ring. The flux is responsible for coherent oscillations on the ring. We suggest the flux may provide the driving force for associations among memories. We applied our theory to rapid-eye movement sleep cycle. We identified the key regulation factors for function through global sensitivity analysis of landscape topography against wirings, which are in good agreements with experiments.neural circuits | free energy | entropy production | nonequilibrium thermodynamics A grand goal of biology is to understand the function of the human brain. The brain is a complex dynamical system (1-6). The individual neurons can develop action potentials and connect with each other through synapses to form the neural circuits. The neural circuits of the brain perpetually generate complex patterns of activity that have been shown to be related with special biological functions, such as learning, long-term associative memory, working memory, olfaction, decision making and thinking (7-9), etc. Many models have been proposed for understanding how neural circuits generate different patterns of activity. HodgkinHuxley model gives a quantitative description of a single neuronal behavior based on the voltage-clamp measurements of the voltage (4). However, various vital functions are carried out by the circuit rather than individual neurons. It is at present still challenging to explore the underlying global natures of the large neural networks built from individual neurons.Hopfield developed a model (5, 6) that makes it possible to explore the global natures of the large neural networks without losing the information of essential biological functions. For symmetric neural circuits, an energy landscape can be constructed that decreas...

show abstract

Critical Slowing and Perception

Friston

Breakspear

Deco

2014

Criticality in Neural Systems

Self Cite

View full text Add to dashboard Cite

It is generally assumed that criticality and metastability underwrite the brain's dynamic repertoire of responses to an inconstant world. From perception to behavior, the ability to respond sensitively to changes in the environment -and to explore alternative hypotheses and policies -seems inherently linked to selforganized criticality. This chapter addresses the relationship between perception and criticality -using recent advances in the theoretical neuroscience of perceptual inference. In short, we will see that (Bayes) optimal inference -on the causes of sensations -necessarily entails a form of criticality -critical slowing -and a balanced synchronization between the sensorium and neuronal dynamics. This means that the optimality principles describing our exchanges with the world may also account for dynamical phenomena that characterize self-organized systems such as the brain.This chapter considers the formal basis of self-organized instabilities that enable perceptual transitions during Bayes-optimal perception. We will consider the perceptual transitions that lead to conscious ignition [1] and how they depend on dynamical instabilities that underlie chaotic itinerancy [2, 3] and self-organized criticality [4][5][6]. We will try to understand these transitions using a dynamical formulation of perception as approximate Bayesian inference. This formulation suggests that perception has an inherent tendency to induce dynamical instabilities that enable the brain to respond sensitively to sensory perturbations. We review the dynamics of perception, in terms of Bayesian optimization (filtering), present a formal conjecture about self-organized instability, and then test this conjecture, using neuronal simulations of perceptual categorization. Perception and Neuronal DynamicsPerceptual categorization speaks to two key dynamical phenomena: transitions from one perceptual state to another and the dynamical mechanisms that permit Criticality in Neural Systems, First Edition. Edited by Dietmar Plenz and Ernst Niebur.

show abstract

Free Energy, Value, and Attractors

Cited by 109 publications

References 90 publications

Active inference and agency: optimal control without cost functions

Active inference and agency: optimal control without cost functions

Nonequilibrium landscape theory of neural networks

Critical Slowing and Perception

Contact Info

Product

Resources

About