Complex Probabilistic Inference

Gershman, Samuel J.; Beck, Jeffrey M.

doi:10.1002/9781119159193.ch33

Cited by 28 publications

(29 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In fact, the intermediate sensory representation is itself often assumed to be the result of an inference process over latent variables in an internal model of the world (Mumford, 1992; Lee and Mumford, 2003; Yuille and Kersten, 2006). This process is naturally formalized as hierarchical inference (Figure 1b) in which feedforward connections communicate the likelihood and feedback communicates the prior or other contextual expectations (Fiser et al, 2010; Pouget et al, 2013; Gershman and Beck, 2016; Tajima et al, 2017; Lange and Haefner, 2020).…”

Section: Resultsmentioning

confidence: 99%

A confirmation bias in perceptual decision-making due to hierarchical approximate inference

Lange

Chattoraj

Beck

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

1Human decisions are known to be systematically biased. A prominent example of such a bias 2 occurs when integrating a sequence of sensory evidence over time. Previous empirical studies di↵er 3 in the nature of the bias they observe, ranging from favoring early evidence (primacy), to favoring 4 late evidence (recency). Here, we present a unifying framework that explains these biases and 5 makes novel psychophysical and neurophysiological predictions. By explicitly modeling both the 6 approximate and the hierarchical nature of inference in the brain, we show that temporal biases 7 depend on the balance between "sensory information" and "category information" in the stimulus. 8Finally, we present new data from a human psychophysics task that confirm that temporal biases 9 can be robustly changed within subjects as predicted by our models. 10Imagine a doctor trying to infer the cause of a patient's symptoms from an x-ray image. Unsure 12 about the evidence in the image, she asks a radiologist for a second opinion. If she tells the 13 radiologist her suspicion, she may bias his report. If she does not, he may not detect a faint 14 diagnostic pattern. As a result, if the evidence in the image is hard to detect or ambiguous, 15 the radiologist's second opinion, and hence the final diagnosis, may be swayed by the doctor's 16 initial hypothesis. The problem faced by these doctors exemplifies the di culty of hierarchical 17 inference: each doctor's suspicion both informs and is informed by their collective diagnosis. If 18 they are not careful, their diagnosis may fall prey to circular reasoning. The brain faces a similar 19 problem during perceptual decision-making: any decision-making area combines sequential signals 20 from sensory brain areas, not directly from sensory input, just as the doctors' consensus is based 21 on their individual diagnoses rather than on the evidence per se. If sensory signals in the brain 22 themselves reflect inferences that combine both prior expectations and sensory evidence, we suggest 23 that this can then lead to an observable perceptual confirmation bias (Nickerson, 1998). 24 We formalize this idea in the context of approximate Bayesian inference and classic evidence-25 integration tasks in which a range of biases has been observed and for which a unifying explanation 26 1 is currently lacking. Evidence-integration tasks require subjects to categorize a sequence of inde-27 pendent and identically distributed (iid) draws of stimuli (Gold and Shadlen, 2007; Bogacz et al., 28 2006). Previous normative models of evidence integration hinge on two quantities: the amount of 29 information available on a single stimulus draw and the total number of draws. One might expect, 30 then, that temporal biases should have some canonical form in tasks where these quantities are 31 matched. However, existing studies are heterogeneous, reporting one of three distinct motifs: some 32 find that early evidence is weighted more strongly (a primacy e↵ect) (Kiani et al., 2008; Nienborg 33 and Cu...

show abstract

Section: Resultsmentioning

confidence: 99%

A confirmation bias in perceptual decision-making due to hierarchical approximate inference

Lange

Chattoraj

Beck

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, it is worth noting that these properties follow from the form of the underlying generative model. The challenge is to identify the appropriate generative model that best explains the generative process (or the empirical responses) of interest (Gershman & Beck, 2017). In the Frozen-Lake simulation, by equipping the agents with beliefs about the current context, we were able (via the generative model and its belief updating process) to convert a learning problem into a planning-as-inference problem.…”

Section: Discussionmentioning

confidence: 99%

Active Inference: Demystified and Compared

et al. 2021

View full text Add to dashboard Cite

Active inference is a first principle account of how autonomous agents operate in dynamic, nonstationary environments. This problem is also considered in reinforcement learning, but limited work exists on comparing the two approaches on the same discrete-state environments. In this letter, we provide (1) an accessible overview of the discrete-state formulation of active inference, highlighting natural behaviors in active inference that are generally engineered in reinforcement learning, and (2) an explicit discrete-state comparison between active inference and reinforcement learning on an OpenAI gym baseline. We begin by providing a condensed overview of the active inference literature, in particular viewing the various natural behaviors of active inference agents through the lens of reinforcement learning. We show that by operating in a pure belief-based setting, active inference agents can carry out epistemic exploration—and account for uncertainty about their environment—in a Bayes-optimal fashion. Furthermore, we show that the reliance on an explicit reward signal in reinforcement learning is removed in active inference, where reward can simply be treated as another observation we have a preference over; even in the total absence of rewards, agent behaviors are learned through preference learning. We make these properties explicit by showing two scenarios in which active inference agents can infer behaviors in reward-free environments compared to both Q-learning and Bayesian model-based reinforcement learning agents and by placing zero prior preferences over rewards and learning the prior preferences over the observations corresponding to reward. We conclude by noting that this formalism can be applied to more complex settings (e.g., robotic arm movement, Atari games) if appropriate generative models can be formulated. In short, we aim to demystify the behavior of active inference agents by presenting an accessible discrete state-space and time formulation and demonstrate these behaviors in a OpenAI gym environment, alongside reinforcement learning agents.

show abstract

“…This means that if we are going to consider expressive generative models, we will need to also consider approximate inference. Historically, approximate inference algorithms have fallen into two families (Gershman and Beck, 2017 ). One family, Monte Carlo algorithms, approximates the posterior via stochastic simulation.…”

Section: Generative Models: Explicit and Implicitmentioning

confidence: 99%

The Generative Adversarial Brain

Gershman

2019

Front. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

The idea that the brain learns generative models of the world has been widely promulgated. Most approaches have assumed that the brain learns an explicit density model that assigns a probability to each possible state of the world. However, explicit density models are difficult to learn, requiring approximate inference techniques that may find poor solutions. An alternative approach is to learn an implicit density model that can sample from the generative model without evaluating the probabilities of those samples. The implicit model can be trained to fool a discriminator into believing that the samples are real. This is the idea behind generative adversarial algorithms, which have proven adept at learning realistic generative models. This paper develops an adversarial framework for probabilistic computation in the brain. It first considers how generative adversarial algorithms overcome some of the problems that vex prior theories based on explicit density models. It then discusses the psychological and neural evidence for this framework, as well as how the breakdown of the generator and discriminator could lead to delusions observed in some mental disorders.

show abstract

Complex Probabilistic Inference

Cited by 28 publications

References 58 publications

A confirmation bias in perceptual decision-making due to hierarchical approximate inference

A confirmation bias in perceptual decision-making due to hierarchical approximate inference

Active Inference: Demystified and Compared

The Generative Adversarial Brain

Contact Info

Product

Resources

About