Multisensory cortical processing of object shape and its relation to mental imagery

Abstract

Here, we used functional magnetic resonance imaging to investigate the multisensory processing of object shape in the human cerebral cortex and explored the role of mental imagery in such processing. Regions active bilaterally during both visual and haptic shape perception, relative to texture perception in the respective modality, included parts of the superior parietal gyrus, the anterior intraparietal sulcus, and the lateral occipital complex. Of these bimodal regions, the lateral occipital complexes preferred visual over haptic stimuli, whereas the parietal areas preferred haptic over visual stimuli. Whereas most subjects reported little haptic imagery during visual shape perception, experiences of visual imagery during haptic shape perception were common. Across subjects, ratings of the vividness of visual imagery strongly predicted the amount of haptic shape-selective activity in the right, but not in the left, lateral occipital complex. Thus, visual imagery appears to contribute to activation of some, but not all, visual cortical areas during haptic perception.

Effective connectivity during haptic perception: A study using Granger causality analysis of functional magnetic resonance imaging data

“…However, the mechanisms underlying such cross-modal recruitment of visual cortex remain uncertain. One idea is that visual imagery could be responsible (Sathian et al, 1997;Sathian & Zangaladze, 2001;Stoesz et al, 2003;Zangaladze et al, 1999;Zhang et al, 2004), whereas other work argues in favor of a common multisensory representation that is engaged by both visual and tactile processing (Amedi et al, 2001(Amedi et al, , 2002James et al, 2002;Lacey et al, 2007). The visual imagery explanation implies involvement of top-down connections from prefrontal or posterior parietal cortex into visual cortical areas (Mechelli et al, 2004), while a multisensory representation might be derived from bottom-up tactile inputs into "visual" cortical areas.…”

“…This underscores the complexity of the underlying circuits and suggests that the traditional separations between sensory and motor areas, or between somatosensory, visual and multisensory regions break down during natural behavior. Thus, while the LOC and MOC are both in visual cortex, the LOC has been repeatedly shown to be engaged during haptic shape perception (Amedi et al, 2001(Amedi et al, , 2002James et al, 2002;Stilla and Sathian, 2007;Zhang et al, 2004) and the MOC, which probably corresponds to V2, was first identified by us as responsive during haptic texture perception . Conversely, the PCS, though a classical somatosensory area, is also shapeselective during visual perception .…”

Haptic perception: A tutorial

“…More specifically, judgments of the layout of touched objects and of stimulus motion on the skin have been found to activate areas in the dorsal visual pathway (macrospatial, Kitada et al, 2006;Sathian, Zangaladze, Hoffman, & Grafton, 1997;Stoesz et al, 2003;motion, R. Blake, Sobel, & James, 2004;Hagen, Zald, Thornton, & Pardo, 2002). However, haptic shape perception of 3-D objects activates the ventral visual pathway (e.g., Amedi, Jacobson, Hendler, Malach, & Zohary, 2002;Amedi, Malach, Hendler, Peled, & Zohary, 2001;Deibert, Kraut, Kremen, & Hart, 1999;James et al, 2002;James, Servos, Kilgour, Huh, & Lederman, 2006;Kitada, Johnsrude, Kochiyama, & Lederman, 2009;Malach et al, 1995;Pietrini et al, 2004;Reed, Shoham, & Halgren, 2004;Stoeckel et al, 2003;Zhang, Weisser, Stilla, Prather, & Sathian, 2004).…”

“…Rapid reversibility of this phenomenon, as demonstrated by Merabet et al (2008), suggests that the effect of visual deprivation may release inhibition that would otherwise be present. been obtained in several tasks involving the tactile perception of grating orientation (e.g., Sathian & Zangaladaze, 2001;Sathian et al, 1997 [see Figure 13]; Zangaladze, Epstein, Grafton, & Sathian, 1999;Zhang et al, 2004) and the haptic recognition of common objects depicted in 2-D raised-outline drawings (Lederman, Klatzky, Chataway, & Summers, 1990). However, visual imagery is by no means necessary, as shown in an fMRI study that compared haptic, visual, and visually imaged identification of specific exemplars of 3-D plaster casts of different body parts (Kitada et al, 2009).…”

Are visual texture-selective areas recruited during haptic texture discrimination?

“…Examinations of haptic perception have found that visually shape-selective regions of the ventral stream (LOC) are also activated when participants investigate object shape via touch Allen and Humphreys, 2009;Grefkes et al, 2002;James et al, 2002;Naumer et al, 2010;Pietrini et al, 2004;Stilla and Sathian, 2008;Tal and Amedi, 2009;Zhang et al, 2004). Importantly, asked participants both to palpate a number of objects and view images of the same objects while inside the fMRI scanner.…”