Author Correction: Multisensory task demands temporally extend the causal requirement for visual cortex in perception

Lohuis, Matthijs N Oude; Pie, Jean L.; Marchesi, Pietro; Montijn, Jorrit S.; Kock, Christiaan P.J. de; Pennartz, Cyriel M. A.; Olcese, Umberto

doi:10.1038/s41467-022-31518-7

Cited by 4 publications

(27 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In combination with work on encoding of errors in superficial, and representations in deep cortical layers (3,63,76,77), area-and layer-wise recordings of characteristic timescales could lead to a better understanding of cortical microcircuits underlying predictive coding. Layer-wise investigations also show distinct patterns of feedforward and feedback connectivity (78) and information processing (79). Only with knowledge about these microcircuits, models of finer granularity can be constructed.…”

Section: Hypotheses On the Neural Circuitry Of Predictive Codingmentioning

confidence: 99%

Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception

Brucklacher

Bohté

Mejías

et al. 2022

Preprint

View full text Add to dashboard Cite

The ventral visual processing hierarchy of the cortex needs to fulfill at least two key functions: Perceived objects must be mapped to high-level representations invariantly of the precise viewing conditions, and a generative model must be learned that allows, for instance, to fill in occluded information guided by visual experience. Here, we show how a multilayered predictive coding network can learn to recognize objects from the bottom up and to generate specific representations via a top-down pathway through a single learning rule: the local minimization of prediction errors. Trained on sequences of continuously transformed objects, neurons in the highest network area become tuned to object identity invariant of precise position, comparable to inferotemporal neurons in macaques. Drawing on this, the dynamic properties of invariant object representations reproduce experimentally observed hierarchies of timescales from low to high levels of the ventral processing stream. The predicted faster decorrelation of error-neuron activity compared to representation neurons is of relevance for the experimental search for neural correlates of prediction errors. Lastly, the generative capacity of the network is confirmed by reconstructing specific object images, robust to partial occlusion of the inputs. By learning invariance from temporal continuity within a generative model, the network goes beyond purely input-reconstructing models of predictive coding and generalizes the framework to moving inputs in a more biologically plausible way than self-supervised networks with non-local error-backpropagation.Author SummaryNeurons in the inferotemporal cortex of primates respond to images of complex objects independent of position, rotational angle, or size. While feedforward models of visual perception such as deep neural networks can explain this, they fail to account for the use of top-down information, for example when sensory evidence is scarce. Here, we address the question of how the neuronal networks in the brain learn both bottom-up and top-down processing without artificial supervision signals. Building on previous work that explains vision as a process of iteratively improving predictions, learning in the predictive coding network is driven by the local minimization of prediction errors. When trained on sequences of moving inputs, the network learns both invariant high-level representations comparable to those in the inferotemporal cortex of primates, and a generative model capable of reconstructing whole objects from partially occluded input images in agreement with experimental recordings from early visual areas. Advancing the search for experimental hallmarks of prediction errors, we find that error neurons in the network change their activity on a shorter timescale than representation neurons.

show abstract

Section: Hypotheses On the Neural Circuitry Of Predictive Codingmentioning

confidence: 99%

Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception

Brucklacher

Bohté

Mejías

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Several other non-sensory sources of information also exert an influence on V1 by modulating visual responses. This is the case, for instance, for arousal [56][57][58], locomotion [59][60][61][62][63], attention [62,[64][65][66] and context-dependent modulation [41,62,[64][65][66][67][68][69]. With 'context', we mean factors that influence how information about single-sensory stimuli is interpreted.…”

Section: (B) Non-sensory Representations In V1mentioning

confidence: 99%

“…(iii) The earliest moment after stimulus onset at which behavioural report (Hit or Miss) is encoded in V1 neurons varies, for the same visual stimulus as a function of task contingency, with subjectively more complex settings associated with later onsets (see also (ii)), which are tightly correlated with, and precede, reaction time. Panels (e-g) have been modified with permission from Oude Lohuis et al [41].…”

Section: (B) Non-sensory Representations In V1mentioning

confidence: 99%

“…After learning such a task, V1 displays a large wave of activity that comes after an early-latency, sensory-related peak in V1 activity. This 'second bump' is understood as report-related activity resulting from recurrent cortical feedback [78,81,82] and is tightly locked to and anticipates the motor response [41,78,81] (figure 1e-g). This widespread wave of activity supposedly originates in pre-motor areas [47,77,82,83] and, upon reaching V1, can even exceed early visually evoked responses in amplitude [41,78].…”

Section: (B) Non-sensory Representations In V1mentioning

confidence: 99%

“…This 'second bump' is understood as report-related activity resulting from recurrent cortical feedback [78,81,82] and is tightly locked to and anticipates the motor response [41,78,81] (figure 1e-g). This widespread wave of activity supposedly originates in pre-motor areas [47,77,82,83] and, upon reaching V1, can even exceed early visually evoked responses in amplitude [41,78]. Yet, its role in visual processing remains poorly understood, because it is not causally involved in visual processing per se [81], even if its onset determines for how long V1 itself is required for visual decision making [41].…”

Section: (B) Non-sensory Representations In V1mentioning

confidence: 99%

See 2 more Smart Citations

How ‘visual’ is the visual cortex? The interactions between the visual cortex and other sensory, motivational and motor systems as enabling factors for visual perception

Pennartz,

Oude Lohuis,

Olcese

2023

Phil. Trans. R. Soc. B

Self Cite

View full text Add to dashboard Cite

The definition of the visual cortex is primarily based on the evidence that lesions of this area impair visual perception. However, this does not exclude that the visual cortex may process more information than of retinal origin alone, or that other brain structures contribute to vision. Indeed, research across the past decades has shown that non-visual information, such as neural activity related to reward expectation and value, locomotion, working memory and other sensory modalities, can modulate primary visual cortical responses to retinal inputs. Nevertheless, the function of this non-visual information is poorly understood. Here we review recent evidence, coming primarily from studies in rodents, arguing that non-visual and motor effects in visual cortex play a role in visual processing itself, for instance disentangling direct auditory effects on visual cortex from effects of sound-evoked orofacial movement. These findings are placed in a broader framework casting vision in terms of predictive processing under control of frontal, reward- and motor-related systems. In contrast to the prevalent notion that vision is exclusively constructed by the visual cortical system, we propose that visual percepts are generated by a larger network—the extended visual system—spanning other sensory cortices, supramodal areas and frontal systems. This article is part of the theme issue ‘Decision and control processes in multisensory perception’.

show abstract

The neural and computational architecture of feedback dynamics in mouse cortex during stimulus report

Ciceri,

Lohuis,

Rottschäfer

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Conscious reportability of visual input is associated with a bimodal neural response in primary visual cortex (V1): an early-latency response coupled to stimulus features and a late-latency response coupled to stimulus report or detection. This late wave of activity, central to major theories of consciousness, is thought to be driven by prefrontal cortex (PFC), responsible for "igniting" it. Here we analyzed two electrophysiological studies in mice performing different stimulus detection tasks, and characterize neural activity profiles in three key cortical regions: V1, posterior parietal cortex (PPC) and PFC. We then developed a minimal network model, constrained by known connectivity between these regions, reproducing the spatio-temporal propagation of visual- and report-related activity. Remarkably, while PFC was indeed necessary to generate report-related activity in V1, this occurred only through the mediation of PPC. PPC, and not PFC, had the final veto in enabling the report-related late wave of V1 activity.

show abstract

Author Correction: Multisensory task demands temporally extend the causal requirement for visual cortex in perception

Cited by 4 publications

References 0 publications

Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception

Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception

How ‘visual’ is the visual cortex? The interactions between the visual cortex and other sensory, motivational and motor systems as enabling factors for visual perception

The neural and computational architecture of feedback dynamics in mouse cortex during stimulus report

Contact Info

Product

Resources

About