Conceptual knowledge affects early stages of visual mental imagery and object perception

Abstract

When we imagine an object and when we actually see that object, similar brain regions become active. Yet, the time course of neurocognitive mechanisms that support imagery is still largely unknown. The current view holds that imagery does not share early perceptual mechanisms, but starts with high-level visual representations. However, evidence of early shared mechanisms is difficult to obtain because imagery and perception tasks typically differ in visual input. We therefore tracked electrophysiological brain responses while fully controlling visual input, (1) comparing imagery and perception of objects with varying amounts of associated knowledge, and (2) comparing the time courses of successful and incomplete imagery. Imagery and perception were similarly influenced by knowledge already at early stages, revealing shared mechanisms during low-level visual processing. It follows that imagery is not merely perception in reverse; instead, both are active and constructive processes, based on shared mechanisms starting at surprisingly early stages. Keywords: mental imagery, early visual processing, event-related potentials, semantic knowledge, P1 component MENTAL IMAGERY: TIME COURSE, COGNITIVE MECHANISMS 3 Time course and shared neurocognitive mechanisms of mental imagery and visual perception A growing body of research suggests that seeing something with the mind's eyemental imagery-may not be all that different from seeing something with one's physical eyes. Indeed, imagery and perception recruit overlapping neural circuits, including primary visual areas 1-8 , and the vividness of imagination correlates with the similarity of brain activities accompanying imagery and perception 9 . Predictive processing accounts posit that perception arises from hierarchical Bayesian predictions-essentially imaginations-that are constrained by bottom-up sensory input 10-14 .This theoretical framework is neurally plausible 15-20 and supported by evidence that even early stages of perception are subject to top-down influences 15,[21][22][23][24][25][26][27][28][29][30][31] . This suggests that initial aspects of imagery could be fast enough to generate early top-down effects.This suggestion contrasts with alternative accounts assuming that perception first runs through a strictly hierarchical succession of increasingly complex visual representations, with early stages mainly driven by bottom-up sensory processes. At later stages, recurrent feedback from higher-level brain areas is assumed to enable stabilization of visual representations and, eventually, conscious access 32,33 . Based on this account of perception, recent work has mapped out how visual imagery could follow a reverse hierarchy of activation compared to perception 6,7,[34][35][36] . Under these assumptions, imagery would not rely on early perceptual mechanisms like feature processing but start relatively late, with entire visual representations that bring several levels of the visual hierarchy into concert 6,7,35 . In support of this idea, Dijkstra, et al. 35 found ...

Gabor and Non-Gabor Neural Representations Are Shared between Visual Perception and Mental Imagery

“…For example, when participants were required to complete a mental image scanning task, a free view scanning task, and an iconic image scanning task with measured RT and error rate for each task, it was found that participants could achieve similar performance between scanning mental images and visual perception (Borst & Kosslyn, 2008). Kosslyn and Pearson have proposed that mental imagery resembles weak perception (Pearson, 2019;Pearson & Kosslyn, 2015), based on the similar neural mechanism of sensory processing between them (Maier, Frömer, Rost, Sommer, & Rahman, 2021;Xie, Kaiser, & Cichy, 2020;Dijkstra, Bosch, & van Gerven, 2019;Cichy, Heinzle, & Haynes, 2012;Reddy, Tsuchiya, & Serre, 2010;Stokes, Thompson, Cusack, & Duncan, 2009;Borst & Kosslyn, 2008;Ishai & Sagi, 1995). According to perception anticipation theory (Aitken, Turner, & Kok, 2020;Kok, Jehee, & de Lange, 2012;Sohoglu, Peelle, Carlyon, & Davis, 2012;Kosslyn & Thompson, 2003), expectation or prediction may have some impact on the top-down modulation process of perception (Berger & Ehrsson, 2013;Diekhof et al, 2011;Pearson, Clifford, & Tong, 2008).…”

“…According to perception anticipation theory (Aitken, Turner, & Kok, 2020;Kok, Jehee, & de Lange, 2012;Sohoglu, Peelle, Carlyon, & Davis, 2012;Kosslyn & Thompson, 2003), expectation or prediction may have some impact on the top-down modulation process of perception (Berger & Ehrsson, 2013;Diekhof et al, 2011;Pearson, Clifford, & Tong, 2008). Some fMRI studies have provided evidence that mental imagery and visual perception share brain activity patterns in the early visual cortex (EVC, i.e., V1, V2, and V3) encoding low-level visual features (Maier et al, 2021;Kosslyn & Thompson, 2003) as well as the high-level visual areas encoding category information (Lee, Kravitz, & Baker, 2012;Stokes et al, 2009;O'Craven & Kanwisher, 2000). In earlier years, researchers used positron emission topography and repetitive TMS to demonstrate that EVC is involved in visual imagery (Kosslyn et al, 1999), and recent research with a multi-method approach (fMRI, TMS, and transcranial direct current stimulation) suggests that EVC serves a causative role in visual imagery (Keogh, Bergmann, & Pearson, 2020).…”

The Effect of Virtual Reality Technology on the Imagery Skills and Performance of Target-Based Sports Athletes

“…Imagery and perception cause similar neural activities in the brain (Dijkstra et al, 2017). If we can create realistic images in our minds, our brain will react as if we really see that image (Maier et al, 2020b). In addition, studies show that there is a positive correlation between perception in visual areas and vividness of imagery.…”

Shared and Independent Neural Representation Between Visual Perception and Mental Imagery

“…For example, in a series of behavioral experiments, participants could reach similar performance between scanning mental images and visual perception (Borst & Kosslyn, 2008). Some researchers have proposed that mental imagery resembles weak perception (Pearson, 2019;Pearson & Kosslyn, 2015), based on the similar neural mechanism of sensory processing between them (Borst & Kosslyn, 2008;Cichy et al, 2012;Dijkstra et al, 2019;Ishai & Sagi, 1995;Maier et al, 2020;Reddy et al, 2010;Stokes et al, 2009;Xie et al, 2020), and according to perception anticipation theory (Aitken et al, 2020;Kok et al, 2012;Kosslyn & Thompson, 2003;Sohoglu et al, 2012) it may have some impact on the top-down modulation process of perception (Berger & Ehrsson, 2013;Diekhof et al, 2011;Pearson et al, 2008). Specifically, some functional magnetic resonance imaging (fMRI) studies have provided evidence that mental imagery and visual perception shared brain activity pattern in the early visual cortex (EVC, i.e., V1, V2, and V3) encoding low-level visual features (Kosslyn & Thompson, 2003;Maier et al, 2020) as well as the high visual areas encoding category information (S. H. Lee et al, 2012;O'Craven & Kanwisher, 2000;Stokes et al, 2009).…”

“…Some researchers have proposed that mental imagery resembles weak perception (Pearson, 2019;Pearson & Kosslyn, 2015), based on the similar neural mechanism of sensory processing between them (Borst & Kosslyn, 2008;Cichy et al, 2012;Dijkstra et al, 2019;Ishai & Sagi, 1995;Maier et al, 2020;Reddy et al, 2010;Stokes et al, 2009;Xie et al, 2020), and according to perception anticipation theory (Aitken et al, 2020;Kok et al, 2012;Kosslyn & Thompson, 2003;Sohoglu et al, 2012) it may have some impact on the top-down modulation process of perception (Berger & Ehrsson, 2013;Diekhof et al, 2011;Pearson et al, 2008). Specifically, some functional magnetic resonance imaging (fMRI) studies have provided evidence that mental imagery and visual perception shared brain activity pattern in the early visual cortex (EVC, i.e., V1, V2, and V3) encoding low-level visual features (Kosslyn & Thompson, 2003;Maier et al, 2020) as well as the high visual areas encoding category information (S. H. Lee et al, 2012;O'Craven & Kanwisher, 2000;Stokes et al, 2009). Recently, advances in machine learning approaches have been used in combination with fMRI data to provide a novel viewpoint for exploring the specific shared neural representation between visual perception and mental imagery (Albers et al, 2013;Horikawa & Kamitani, 2017; S. H. Lee et al, 2012;Pearson et al, 2008;Reddy et al, 2010).…”