Seeing Patterns in Randomness: A Computational Model of Surprise

Abstract

While seemingly a ubiquitous cognitive process, the precise definition and function of surprise remains elusive. Surprise is often conceptualized as being related to improbability or to contrasts with higher probability expectations. In contrast to this probabilistic view, we argue that surprising observations are those that undermine an existing model, implying an alternative causal origin. Surprises are not merely improbable events; instead, they indicate a breakdown in the model being used to quantify probability. We suggest that the heuristic people rely on to detect such anomalous events is randomness deficiency. Specifically, people experience surprise when they identify patterns where their model implies there should only be random noise. Using algorithmic information theory, we present a novel computational theory which formalizes this notion of surprise as randomness deficiency. We also present empirical evidence that people respond to randomness deficiency in their environment and use it to adjust their beliefs about the causal origins of events. The connection between this pattern-detection view of surprise and the literature on learning and interestingness is discussed.

Editors’ Introduction and Review: An Appraisal of Surprise: Tracing the Threads That Stitch It Together

“…As illustrated by several papers in this issue, these three approaches need not be mutually exclusive, and can often be complementary. For example, Macedo and Cardoso (2019) propose a Bayesian model, and Maguire, Moser, Maguire, and Keane (2019) propose a potentially complementary information-theoretic account that can deal with surprises that arise when one has no specific expectations (hence no Bayesian prior). In addition, Sim and Xu (2019) discuss both Bayesian (e.g., T egl as et al, 2011) and information-theoretic (e.g., Kidd et al, 2014) approaches that have informed work on infants' reasoning.…”

Why the Conjunction Effect Is Rarely a Fallacy: How Learning Influences Uncertainty and the Conjunction Rule

“…Both Baldi and Itti ( 2010 ) and Maguire et al ( 2018 ) have provided converging theories of surprise which model how people learn from subjectively uncertain outcomes, one based on Bayesian probability and the other on algorithmic information theory.…”

“…One issue with Baldi and Itti's ( 2010 ) formulation of surprise is that computing it requires identifying a set of relevant hypotheses. Maguire et al ( 2018 ) provide a more generalized theory of surprise based on algorithmic information theory (AIT; see Li and Vitányi, 2008 ). Whereas Baldi and Itti's ( 2010 ) model expresses the informativeness of an observation relative to a pre-defined set of competing hypotheses, Maguire et al's model expresses subjective informativeness in terms of the universal likelihood measure of randomness deficiency (see also Maguire et al, 2014 , 2016 ).…”

“…Maguire et al's ( 2018 ) idea is as follows: surprise is the normalised difference between the probabilistic point of view, which treats observations as independent (i.e., it assumes that the uncertainty is objective), and the computational point of view, which gives the shortest possible description of a set of observations (i.e., it allows for the possibility of learning). For example, from the probabilistic point of view the coin tossing outcome string “HHHHHHHHHHHHHHHHHHHH” requires 20 bits to encode, because each bit is independent.…”

“…Rather than assuming a fixed model of reality, people are sensitive to any patterns that link outcomes together, thereby suggesting the presence of a superior underlying explanatory model (e.g., is there something I don't know that other people might know? ; see Schmidhuber, 2009 ; Maguire et al, 2018 ). This idea, that outcomes can be subjectively informative, clashes with a core assumption of probability theory, namely that successive outcomes should be treated as independent of each other.…”