A Mixture of Delta-Rules Approximation to Bayesian Inference in Change-Point Problems

Abstract

Error-driven learning rules have received considerable attention because of their close relationships to both optimal theory and neurobiological mechanisms. However, basic forms of these rules are effective under only a restricted set of conditions in which the environment is stable. Recent studies have defined optimal solutions to learning problems in more general, potentially unstable, environments, but the relevance of these complex mathematical solutions to how the brain solves these problems remains unclear. Here, we show that one such Bayesian solution can be approximated by a computationally straightforward mixture of simple error-driven ‘Delta’ rules. This simpler model can make effective inferences in a dynamic environment and matches human performance on a predictive-inference task using a mixture of a small number of Delta rules. This model represents an important conceptual advance in our understanding of how the brain can use relatively simple computations to make nearly optimal inferences in a dynamic world.

Formation of a long-term memory for visuomotor adaptation following only a few trials of practice

“…Thus, nominally, the reward prediction errors during adaptation would be negative (i.e., reward is less than expected), because performance is worse under the perturbation compared with at baseline. Whether subjects interpret a given action as an improvement (a positive reward prediction error) or continued failure (a negative reward prediction error) may depend on whether they detect that a change point had occurred in the experiment following the rotation onset (Wilson et al 2013). If the imposed rotation is interpreted as a change, actions and/or strategies that reduce the initially large errors experienced after the onset of the perturbation may be associated with a positive reward prediction error and thus may be remembered.…”

Pharmacological Fingerprints of Contextual Uncertainty

“…Hierarchical Bayesian models have proven powerful for explaining the adaptation of behaviour to probabilistic contexts in volatile environments [1,15,17,70,72–74,84,104,105]. Here we developed a novel instantiation of the HGF model [9] with a focus on TMs and two components: a three-level perceptual model and a response model (Fig 3A).…”

Time discounting and time preference in animals: A critical review

“…This set includes tasks based on the basic problems of foraging theory, including the patch-leaving problem, the diet selection problem, the central place foraging problem, and so forth (Stephens & Krebs, 1986). It also includes stopping problems and other classic optimization problems, such as the k-arm bandit problems, horizon problems, and change point detection problems (Pearson, Hayden, Raghavachari, & Platt, 2009;Wilson, Geana, White, Ludvig, & Cohen, 2014;Wilson, Nassar, & Gold, 2013). Indeed, it may also include variants of the intertemporal choice task in which the postreward delays are clearly cued (Pearson et al, 2010).…”