Spontaneously emerging internal models of visual sequences combine abstract and event-specific information in the prefrontal cortex

Abstract

Theories of predictive coding hypothesize that cortical networks learn internal models of environmental regularities to generate expectations that are constantly compared with sensory inputs. The prefrontal cortex (PFC) is thought to be critical for predictive coding. Here, we show how prefrontal neuronal ensembles encode a detailed internal model of sequences of visual events and their violations. We recorded PFC ensembles in a visual local-global sequence paradigm probing low and higher-order predictions and mismatches. PFC ensembles formed distributed, overlapping representations for all aspects of the dynamically unfolding sequences, including information about image identity as well as abstract information about ordinal position, anticipated sequence pattern, mismatches to local and global structure, and model updates. Model and mismatch signals were mixed in the same ensembles, suggesting a revision of predictive processing models that consider segregated processing. We conclude that overlapping prefrontal ensembles may collectively encode all aspects of an ongoing visual experience, including anticipation, perception, and surprise.

Brainstem fMRI signaling of surprise across different types of deviant stimuli

“…Previous studies showed that the global deviance detection vanishes in patients with disorders of consciousness (Faugeras et al, 2012), when healthy subjects fall asleep (Strauss et al, 2015), and when they are not aware (or do not pay attention to) the task structure (Quirins et al, 2018). Interestingly, the effect of global deviance (notably rare xxxxx patterns) is more difficult to detect, and with a reduced extent, in brain recordings of macaque monkeys (Bellet et al, 2021; Chao et al, 2018; Jiang et al, 2022; Uhrig et al, 2014), for which global deviants are not behavioral relevant and thus potentially not attended, compared to healthy human participants who are told about the existence of global deviants and often asked to count them (Bekinschtein et al, 2009; Karoui et al, 2014; Quirins et al, 2018; Strauss et al, 2015; Wacongne et al, 2011). Here, we also asked participants to count the global deviants, which probably enhanced their detection and the associated brain responses.…”

Monkey Prefrontal Cortex Learns to Minimize Sequence Prediction Error

“…As a starting point, the free energy principle (Friston, 2010) proposes conceptually that the brain minimizes prediction error, either by learning to better predict the world, or by acting to minimize the difference between intended and actual outcomes (Friston et al, 2011; Zarr and Brown, 2023). There is empirical support for the notion that monkey prefrontal cortex encodes sequences (Bellet et al, 2021; Averbeck et al, 2006), as well as errors in sequence prediction as it relates to reward (Oemisch et al, 2019). However, how such signals are used to govern learning in prefrontal cortex is not fully understood.…”

Different subregions of monkey lateral prefrontal cortex respond to abstract sequences and their components

“…FMRI studies in macaques have shown VLPFC activity during auditory sequential tasks and sequence deviants (Wang et al, 2015;Vergnieux and Vogels, 2020). Studies using electrophysiology provided evidence for the representation of generalizable sequential structures and changes to these structures in neuronal population responses within VLPFC (Esmailpour et al, 2023;Bellet et al, 2024). VLPFC also responds to non-sequential information that shares similar features with sequential tasks, such as prediction error (Uhrig et al, 2014;Chao et al, 2018) and responses to infrequent ("oddball") items (Uhrig et al, 2014;Suda et al, 2022).…”