An information-theoretic perspective on the costs of cognition

Abstract

In statistics and machine learning, model accuracy is traded off with complexity, which can be viewed as the amount of information extracted from the data. Here, we discuss how cognitive costs can be expressed in terms of similar information costs, i.e. as a function of the amount of information required to update a person's prior knowledge (or internal model) to effectively solve a task. We then examine the theoretical consequences that ensue from this assumption. This framework naturally explains why some tasks -for example, unfamiliar or dual tasks -are costly and permits to quantify these costs using information-theoretic measures. Finally, we discuss brain implementation of this principle and show that subjective cognitive costs can originate either from local or global capacity limitations on information processing or from increased rate of metabolic alterations. These views shed light on the potential adaptive value of cost-avoidance mechanisms.

Eye pupil signals information gain

“…manipulation of working memory, planning, mind-wandering, mental imagery or offline learning. Therefore, tonic pupil size would increase when cognitive activity occurs out of sync with task events [76], hence decreasing limited cognitive resources available for main task [61], leading to distractibility and exploratory behaviour, but it would also increase during demanding covert computations on working memory [81,82,25]. However, confirming this hypothesis requires quantifying out-of-sync information , which follow the same trend (increasing with uncertainty) as the pupil size reported in the original study (black dots).…”

“…Another common findings in the literature is that pupil size varies as a function of task demands and subject's engagement in the task, suggesting the view that pupillary dilation indexes mental effort [2,4,5,68,69,70]. We have recently proposed that mental effort too can be quantified as the average KL divergence between prior and posterior beliefs [61]. Effortful tasks often include large number of associations between stimuli and responses, resulting in low prior beliefs for each association and requiring large updates in order to reach precise posterior beliefs (e.g.…”

Pupil-Linked Arousal Responds to Unconscious Surprisal

“…However, pupil size is also affected by global brain arousal state (Bradley et al, 2008;Reimer et al, 2016), which is modulated by a wide range of cognitive factors (Hess and Polt, 1964;Beatty and Lucero-Wagoner, 2000;Bradley et al, 2008;Mathot et al, 2014). The fundamental, causal link between cognition and arousal remains unknown but authors have hypothesized that uncertainty, and its reduction through brain processing, may be the common denominator of arousal responses to cognition (Yu and Dayan, 2005;Preuschoff et al, 2011;Zénon et al, 2019). In agreement with this view, pupil size has been shown to respond to self-information, or surprisal, which can be defined as the negative log probability of an event, i.e., how unexpected an observation is (Friedman et al, 1973;Raisig et al, 2010;Preuschoff et al, 2011;Nassar et al, 2012;O'Reilly et al, 2013;Damsma and van Rijn, 2017).…”

Electroencephalographic Correlates of Temporal Bayesian Belief Updating and Surprise

“…More specifically, the Bayesian brain hypothesis supposes that the brain represents information as posterior probabilities, which are derived by integrating likelihoods of newly gained information with prior probabilities, that is, previously held (prior) beliefs. A general brain function hence is to update prior beliefs to build accurate predictive models that minimize discrepancies (i.e., surprise) -which may impose cognitive costs (Zénon et al, 2019) -between newly gained information and prior beliefs.…”