Surprise: a unified theory and experimental predictions

Abstract

Surprising events trigger measurable brain activity and influence human behavior by affecting learning, memory, and decision-making. Currently there is, however, no consensus on the definition of surprise. Here we identify 16 mathematical definitions of surprise in a unifying framework, show how these definitions relate to each other, and prove under what conditions they are indistinguishable. We classify these surprise measures into four main categories: (i) change-point detection surprise, (ii) information gain surprise, (iii) prediction surprise, and (iv) confidence-correction surprise. We design experimental paradigms where different categories make different predictions: we show that surprise-modulation of the speed of learning leads to sensible adaptive behavior only for change-point detection surprise whereas surprise-seeking leads to sensible exploration strategies only for information gain surprise. However, since neither change-point detection surprise nor information gain surprise perfectly reflect the definition of ‘surprise’ in natural language, a combination of prediction surprise and confidence-correction surprise is needed to capture intuitive aspects of surprise perception. We formalize this combination in a new definition of surprise with testable experimental predictions. We conclude that there cannot be a single surprise measure with all functions and properties previously attributed to surprise. Consequently, we postulate that multiple neural mechanisms exist to detect and signal different aspects of surprise.Author noteAM is grateful to Vasiliki Liakoni, Martin Barry, and Valentin Schmutz for many useful discussions in the course of the last few years, and to Andrew Barto for insightful discussions through and after EPFL Neuro Symposium 2021 on “Surprise, Curiosity and Reward: from Neuroscience to AI”. We thank K. Robbins and collaborators for their publicly available experimental data (Robbins et al., 2018). All code needed to reproduce the results reported here will be made publicly available after publication acceptance. This research was supported by Swiss National Science Foundation (no. 200020_184615). Correspondence concerning this article should be addressed to Alireza Modirshanechi, School of Computer and Communication Sciences and School of Life Sciences, EPFL, Lausanne, Switzerland. E-mail: alireza.modirshanechi@epfl.ch.

The curse of optimism: a persistent distraction by novelty

“…3B3). These observations confirm the inefficiency of seeking surprise and the efficiency of seeking information gain in dealing with noise 25,29 .…”

“…4 and Fig. 6A), we define the intrinsic reward function as the Shannon surprise 29 where p ( t ) ( s ′| s, a ) is defined in Eq. 6.…”

Error-based or target-based? A unified framework for learning in recurrent spiking networks

“…The sign of this can be observed in the fragmented panorama of different theories and models proposed in the literature. In recent years, theoretical neuroscientists have formulated new frameworks attempting at providing more general explanations to aspects concerning intelligence and learning [41,42]. In this work we contribute to this generalization effort by providing a general framework that is capable to account for different learning approaches by modulating two sensible parameters, the of the feedback error propagation R and the tolerance to precise spike timing τ ?…”

Semantic Surprise Predicts the N400 Brain Potential

“…Following earlier work on the N400 (Rabovsky et al, 2018), we implemented our semantic surprise measure as the Bayesian surprise, which indexes the extent to which a stimulus causes the learner to update its beliefs (Modirshanechi et al, 2021). If, for example, a learner has previously been presented with only land animals, seeing another land animal will not cause a large adjustment to the learner’s beliefs, resulting in a small BS.…”

“…Seeing a tool will, however, prompt the learner to reconsider the probabilities of the respective semantic categories, and will thus cause a larger Bayesian surprise. The BS is defined as the Kullback-Leibler (KL) divergence between the model’s beliefs prior to, and after, the presentation of a stimulus (Modirshanechi et al, 2021). An example of how BS varies over a sequence of stimuli in the roving paradigm is displayed in the upper panel of Figure 1.…”