pymdp: A Python library for active inference in discrete state spaces

Heins, Conor; Millidge, Beren; Demekas, Daphne; Klein, Brennan; Friston, Karl; Couzin, Iain; Tschantz, Alexander

doi:10.48550/arxiv.2201.03904

Cited by 9 publications

(13 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that our results also hold under the active inference framework (23,58). Canonical versions of active inference contain a generative model that is very similar to ours (Hidden Markov model, HMM), with the difference of an additional layer that models an agent's actions, or choices (59). Actions then influence the transitions between states so that more likely, or expected states under the current observations, are reached.…”

Section: Chronic Pain As Biased Inferencesupporting

confidence: 63%

A Bayesian model for chronic pain

2022

View full text Add to dashboard Cite

The perceiving mind constructs our coherent and embodied experience of the world from noisy, ambiguous and multi-modal sensory information. In this paper, we adopt the perspective that the experience of pain may similarly be the result of a probabilistic, inferential process. Prior beliefs about pain, learned from past experiences, are combined with incoming sensory information in a Bayesian manner to give rise to pain perception. Chronic pain emerges when prior beliefs and likelihoods are biased towards inferring pain from a wide range of sensory data that would otherwise be perceived as harmless. We present a computational model of interoceptive inference and pain experience. It is based on a Bayesian graphical network which comprises a hidden layer, representing the inferred pain state; and an observable layer, representing current sensory information. Within the hidden layer, pain states are inferred from a combination of priors p(pain), transition probabilities between hidden states p(paint+1∣paint) and likelihoods of certain observations p(sensation∣pain). Using variational inference and free-energy minimization, the model is able to learn from observations over time. By systematically manipulating parameter settings, we demonstrate that the model is capable of reproducing key features of both healthy- and chronic pain experience. Drawing on mathematical concepts, we finally simulate treatment resistant chronic pain and discuss mathematically informed treatment options.

show abstract

Section: Chronic Pain As Biased Inferencesupporting

confidence: 63%

A Bayesian model for chronic pain

2022

View full text Add to dashboard Cite

show abstract

“…3. Decision-making: execute the most likely decision a t+1 according to 4; implementation details on generic POMDPs are available in [81,91,260,261]. For more complex simulations of sequential decision-making (e.g., involving hierarchical POMDPs), please see [88,91,223,224,[262][263][264].…”

Section: The Basic Active Inference Algorithmmentioning

confidence: 99%

Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

Barp,

Da Costa,

França

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this chapter, we identify fundamental geometric structures that underlie the problems of sampling, optimisation, inference and adaptive decision-making. Based on this identification, we derive algorithms that exploit these geometric structures to solve these problems efficiently. We show that a wide range of geometric theories emerge naturally in these fields, ranging from measure-preserving processes, information divergences, Poisson geometry, and geometric integration. Specifically, we explain how (i) leveraging the symplectic geometry of Hamiltonian systems enable us to construct (accelerated) sampling and optimisation methods, (ii) the theory of Hilbertian subspaces and Stein operators provides a general methodology to obtain robust estimators, (iii) preserving the information geometry of decision-making yields adaptive agents that perform active inference. Throughout, we emphasise the rich connections between these fields; e.g., inference draws on sampling and optimisation, and adaptive decision-making assesses decisions by inferring their counterfactual consequences. Our exposition provides a conceptual overview of underlying ideas, rather than a technical discussion, which can be found in the references herein.

show abstract

“…Each agent is equipped with the single generative model of opinion formation, as described in the previous sections. All simulations described below were conducted using pymdp, a freely available Python package for performing active inference in discrete state spaces [126].…”

Section: Multi-agent Simulationsmentioning

confidence: 99%

“…) currently being optimised. To find the full, multi-factor posterior Q * (s t ), this equation is iterated across marginals, holding the existing solutions for all other marginals fixed while a particular one is updated [126].…”

Section: Appendix A1 State Estimationmentioning

confidence: 99%

Epistemic Communities under Active Inference

Albarracin

Daphne

Ramstead

et al. 2022

Entropy

Self Cite

View full text Add to dashboard Cite

The spread of ideas is a fundamental concern of today’s news ecology. Understanding the dynamics of the spread of information and its co-option by interested parties is of critical importance. Research on this topic has shown that individuals tend to cluster in echo-chambers and are driven by confirmation bias. In this paper, we leverage the active inference framework to provide an in silico model of confirmation bias and its effect on echo-chamber formation. We build a model based on active inference, where agents tend to sample information in order to justify their own view of reality, which eventually leads to them to have a high degree of certainty about their own beliefs. We show that, once agents have reached a certain level of certainty about their beliefs, it becomes very difficult to get them to change their views. This system of self-confirming beliefs is upheld and reinforced by the evolving relationship between an agent’s beliefs and observations, which over time will continue to provide evidence for their ingrained ideas about the world. The epistemic communities that are consolidated by these shared beliefs, in turn, tend to produce perceptions of reality that reinforce those shared beliefs. We provide an active inference account of this community formation mechanism. We postulate that agents are driven by the epistemic value that they obtain from sampling or observing the behaviours of other agents. Inspired by digital social networks like Twitter, we build a generative model in which agents generate observable social claims or posts (e.g., ‘tweets’) while reading the socially observable claims of other agents that lend support to one of two mutually exclusive abstract topics. Agents can choose which other agent they pay attention to at each timestep, and crucially who they attend to and what they choose to read influences their beliefs about the world. Agents also assess their local network’s perspective, influencing which kinds of posts they expect to see other agents making. The model was built and simulated using the freely available Python package pymdp. The proposed active inference model can reproduce the formation of echo-chambers over social networks, and gives us insight into the cognitive processes that lead to this phenomenon.

show abstract

pymdp: A Python library for active inference in discrete state spaces

Cited by 9 publications

References 44 publications

A Bayesian model for chronic pain

A Bayesian model for chronic pain

Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents

Epistemic Communities under Active Inference

Contact Info

Product

Resources

About