Win-Stay, Lose-Sample: A Simple Sequential Algorithm for Approximating Bayesian Inference

Abstract

People can behave in a way that is consistent with Bayesian models of cognition, despite the fact that performing exact Bayesian inference is computationally challenging. What algorithms could people be using to make this possible? We show that a simple sequential algorithm "Win-Stay, Lose-Sample", inspired by the Win-Stay, Lose-Shift (WSLS) principle, can be used to approximate Bayesian inference. We investigate the behavior of adults and preschoolers on two causal learning tasks to test whether people might use a similar algorithm. These studies use a "mini-microgenetic method", investigating how people sequentially update their beliefs as they encounter new evidence. Experiment 1 investigates a deterministic causal learning scenario and Experiments 2 and 3 examine how people make inferences in a stochastic scenario. The behavior of adults and preschoolers in these experiments is consistent with our Bayesian version of the WSLS principle. This algorithm provides both a practical method for performing Bayesian inference and a new way to understand people's judgments.

Formalizing Neurath’s ship: Approximate algorithms for online causal learning.

“…As with Bramley et al, Bonawitz et al (2014) found that children and adults' online structure judgments exhibited sequential dependence. To account for this they proposed an account of how causal learners might rationally reduce the computational effort of continually reconsidering their model.…”

“…We first formalize causal model inference at the computational level. We then highlight the ways in which past experiments have shown human learning to diverge from the predictions of this idealized account, using these to motivate two causal judgment heuristics proposed in the literature: simple endorsement (Bramley, Lagnado, & Speekenbrink, 2015;Fernbach & Sloman, 2009) and win-stay, lose-sample (Bonawitz, Denison, Gopnik, & Griffiths, 2014) before developing out own Neurath's ship framework for belief change and active learning.…”

“…Win-stay, lose-sample (WSLS) (Bonawitz et al, 2014) predicts that participants stick with their current model b t−1 with probability p(d t |b t−1 , w, c t ) or else draw a sample from the full posterior with probability 1 − p(d t |b t−1 , w, c t ). The fitted version of this model had a single lapse parameter giving…”

Barrier Certificates for Assured Machine Teaching

“…The state-dependent teaching model is also found to be consistent with simple human learning models in cognitive science, including the "win-stay lose-shift" model [8], [9] (e.g., when σ(h ; h) = 0 if h = h and 1 otherwise, the learner prefers to stay at the same hypothesis if it is consistent with the observed data).…”

Asking better questions: How presentation formats influence information search.

“…We begin by introducing the option learning and function learning models of human learning, followed by a discussion of potential sampling strategies. Lastly, we also consider several heuristics that do not build a representation of the world (i.e., no predictions about reward), but also capture aspects of human search behavior (Bonawitz, Denison, Gopnik, & Griffiths, 2014;Raichlen et al, 2014).…”