The brain's predictive prowess revealed in primary visual cortex

Abstract

We humans carry around in our heads rich internal mental models that constitute our construction of the world, and the relation of that world to us. These models can be expressed at multiple levels of abstraction, including beliefs about sensory stimuli and the output of our motor programs, or higher-level beliefs about self. Advancing our understanding of the brain's internal processing states, however challenging, could lead to breakthroughs in understanding states such as dreaming, consciousness, and mental disorders (1). Computational theories propose that internal models are broadcast throughout the brain, including to sensory areas of the brain (2). Techniques to read out internal mental models will deliver insight into how our brains use beliefs or predictions to "construct" the environment. Internal models in higher brain areas are complex though, creating a paradox that they are difficult to constrain if we want to decipher the brain's coding of them. However, when internal models are fed back to sensory cortex, we assume that they are translated into sensory predictions that would intuitively have simpler content. In PNAS, Chong et al. (3) provide empirical evidence needed to drive this theory forward. Using brain reading, they show that the brain constructs new plausible predictions of expected sensory input, and that these predictions can be read out in sensory cortex.There are two information streams that coincide in V1: the high-precision retinal input that is processed with high spatial acuity and the broad, abstract, lessprecise strokes painted by cortical feedback. V1 is retinotopically organized and has small classical feedforward receptive fields. Conversely, feedback conveys information about larger portions of the visual field, giving rise to extraclassical or contextual receptive fields. By eliminating the feed-forward input to the classical receptive field, we can investigate internal signals carried by feedback, which originate from diffuse brain areas (4). There are many challenges that we need to understand, including what kind of features are signaled in feedback. In a novel variation of the apparent motion illusion, Chong et al. (3) used inducer stimuli that allowed them to show that the motion prediction fed back to V1 contained detailed texture information. The information that they were able to read out from V1 was created by the brain's prediction about an object's rotational movement as it moved through the visual field. But, in fact, this experience of rotation was never present in the sensory input to the brain-it was fabricated by the brain out of acquired world knowledge. A previous study used objects rotating in depth to show a similar construction of an unseen (but implied) object position, but only in higher areas of the visual hierarchy (5). A number of other studies have shown that internal processing is contextual, both in higher (6) and lower visual areas (7,8). The innovation of the data of Chong et al. (3) is in showing that internal processing is constructive and dy...

Happily entangled: prediction, emotion, and the embodied mind

“…iii This helps make sense of recent work showing that top-down effects (expectation and context) impact processing even in early visual processing areas such as V1-see Petro et al (2014) Petro and Muckli (2016). Recent work in cognitive neuroscience has begun to suggest some of the detailed ways in which biological brains might implement such multi-level prediction machines -see Bastos et al (2012).…”

Why do we remember? The communicative function of episodic memory

“…What happens when an agent draws on episodic recollection, in the exercise of a complex skill, in the recall of a past event, or for that matter in the social-communicative situations that M&C highlight? We suggest that he or she has recourse to stored internal models (Petro & Muckli 2016;Wolpert et al 1995;1998) of previous actions, which represent the agent in his or her interactions with the environment from the agent's point of view. The stored internal model is the memory trace that is embellished by a process of scenario construction.…”

Contextual modulation of primary visual cortex by auditory signals

“…Feed-forward representations in V1 are of a high spatial resolution, but feedback to V1 is less spatially precise [26] and contains higher-level (more abstract) representations of stimulus spaces [66]. This computation of sensory inputs with top-down influences depends on the instruction of the top-down signal; for example, feedback can reconstruct an absent feedforward stimulus [67,68], enhance an illusory feedforward input [69] or suppress a predictable stimulus [70]. We do not know yet how these two streams of representation are combined in V1, but assume that both are constrained by retinotopy.…”