Retrospective surprise: A computational component for active inference

Abstract

In the free energy principle (FEP), proposed by Friston, it is supposed that agents seek to minimize the “surprise”–the negative log (marginal) likelihood of observations (i.e., sensory stimuli)–given the agents' current belief. This is achieved by minimizing the free energy, which provides an upper bound on the surprise. The FEP has been applied to action selection in a framework called “active inference,” where agents are supposed to select an action so that they minimize the “expected free energy” (EFE). While the FEP and active inference have attracted the attention of researchers in a wide range of fields such as psychology and psychiatry, as well as neuroscience, it is not clear which psychological construct EFE is related to. To facilitate the discussion and interpretation of psychological processes underlying active inference, we introduce a computational component termed the “retrospective (or residual) surprise,” which is the surprise of an observation after updating the belief given the observation itself. We show that the predicted retrospective surprise (PRS) provides a lower bound on EFE: EFE is always larger than PRS. We illustrate the properties of EFE and PRS using examples of inference for a binary hidden cause given a binary observation. Essentially, EFE and PRS show similar behavior; however, in certain situations, they provide different predictions regarding action selection. This study also provides insights into the mechanism of active inference based on EFE.