Sure enough: efficient Bayesian learning and choice

Abstract

Probabilistic decision-making is a general phenomenon in animal behaviour, and has often been interpreted to reflect the relative certainty of animals’ beliefs. Extensive neurological and behavioral results increasingly suggest that animal beliefs may be represented as probability distributions, with explicit accounting of uncertainty. Accordingly, we develop a model that describes decision-making in a manner consistent with this understanding of neuronal function in learning and conditioning. This first-order Markov, recursive Bayesian algorithm is as parsimonious as its minimalist point-estimate, Rescorla-Wagner analogue. We show that the Bayesian algorithm can reproduce naturalistic patterns of probabilistic foraging, in simulations of an experiment in bumblebees. We go on to show that the Bayesian algorithm can efficiently describe the behavior of several heuristic models of decision-making, and is consistent with the ubiquitous variation in choice that we observe within and between individuals in implementing heuristic decision-making. By describing learning and decision-making in a single Bayesian framework, we believe we can realistically unify descriptions of behaviour across contexts and organisms. A unified cognitive model of this kind may facilitate descriptions of behavioural evolution.

Is exploration a metric for information gathering? Attraction to novelty and plasticity in black‐capped chickadees

“…Our work suggests that individuals that show a higher overall exploration score in a novel environment test are more likely to sample novel over familiar information compared with slower explorers. Our results thus support the information‐gathering hypothesis (IGS), where more exploratory individuals are expected to choose novel over known resources, because novel items in the environment harbor more uncertainty than previously sampled items (Foley & Marjoram, ; Inglis, Langton, Forkman, & Lazarus, ). In Arvidsson and Matthysen ()'s work, wild‐caught great tits ( Parus major ) were first provided with a binary choice between a profitable and non‐profitable feeder.…”

“…By gaining information about its environment, an animal can reduce uncertainty which increases its likelihood of making more adapted choices about mates, predation, or resources (Mathot, Wright, Kempenaers, & Dingemanse, ; Schmidt, Dall, & van Gils, ; Shettleworth, Krebs, Stephens, & Gibbon, ; Stephens & Krebs, ). Uncertainty is caused by an animal's lack of knowledge about some aspect of its environment and should thus be maximum when encountering a new stimuli or location (Foley & Marjoram, ; Inglis, ). Individuals can gain knowledge via exploration, often defined as the acquisition of information as the animal moves through the environment (Verbeek, Drent, & Wiepkema, ).…”

Bayesian updating during development predicts genotypic differences in plasticity

“…In recent decades, psychologists and behavioral ecologists have used Bayesian approaches to model a specific type of developmental plasticity: learning. Bayesian models have been successfully used to study conditioning, associative and reinforcement learning, perceptual learning, causal learning, and decision-making in humans and animals (Holyoak and Cheng 2011;Trimmer et al 2011;Series and Seitz 2013;Soto et al 2014;Gershman 2015;Foley and Marjoram 2017;Gershman 2017). As is the case for more-general models of Bayesian development, these models assume that particular types of experience encourage changes in behavior because those experiences provide information about conditions in the external environment.…”

“…; Series and Seitz ; Soto et al. ; Gershman ; Foley and Marjoram ; Gershman ). As is the case for more‐general models of Bayesian development, these models assume that particular types of experience encourage changes in behavior because those experiences provide information about conditions in the external environment.…”

Two-dimensional reward evaluation in mice

“…In an earlier version of the foraging model, mice started without knowledge of the reward properties and learned through Bayesian updating (Foley and Marjoram 2017). To focus on post-acquisition performance we removed the first 150 visits, like we did with the empirical data.…”