Visual, haptic and bimodal scene perception: Evidence for a unitary representation

Abstract

Participants studied seven meaningful scene-regions bordered by removable boundaries (30 s each). In Experiment 1 (N=80) participants used visual or haptic exploration and then minutes later, reconstructed boundary position using the same or the alternate modality. Participants in all groups shifted boundary placement outward (boundary extension), but visual study yielded the greater error. Critically, this modality-specific difference in boundary extension transferred without cost in the cross-modal conditions, suggesting a functionally unitary scene representation. In Experiment 2 (N= 20), bimodal study led to boundary extension that did not differ from haptic exploration alone, suggesting that bimodal spatial memory was constrained by the more “conservative” haptic modality. In Experiment 3 (N=20), as in picture studies, boundary memory was tested 30 s after viewing each scene-region and as with pictures, boundary extension still occurred. Results suggest that scene representation is organized around an amodal spatial core that organizes bottom-up information from multiple modalities in combination with top-down expectations about the surrounding world.

Tactile motion lacks momentum

“…Another illusion in the visual and haptic modality possibly related to RM is boundary extension, where visual scenes are remembered beyond their physical boundary, suggesting topdown influences (Intraub, 2004;Intraub, Morelli, & Gagnier, 2015).…”

Anticipatory scene representation in preschool children's recall and recognition memory

“…a ‘kitchen counter scene’ bounded by a frame), following either visual or haptic exploration (without vision) of the framed region. Participants, under these circumstances, remembered having seen or felt beyond the boundaries of the studied region, depending on modality (Intraub, ; Intraub, Morelli & Gagnier, ; and Mullally, Intraub & Maguire, ). When a ‘haptic expert’ – a woman deaf and blind since early life – explored the regions with her hands, she too experienced boundary extension, remembering having felt beyond the original view‐boundaries in the absence of a corresponding sensory input (Intraub, ).…”

Effect of contextual knowledge on spatial layout extrapolation

“…Indeed, this phenomenon has also been generalized to haptic, both with blindfolded sighted participants and with a Bhaptic expert^(i.e., a young woman blind and deaf since early life; Intraub, 2004). This suggests that the phenomenon is a fundamental aspect of scene representation, in which visual and haptic scene representations are even supported by a unitary representation (Intraub, Morelli, & Gagnier, 2015). Similarly, BE has been observed across the lifespan, as much with 3-or 4-month-old infants as with the elderly (Quinn & Intraub, 2007;Seamon, Schleger, Heister, Landau & Blumenthal, 2002).…”

“…Different sources of information Bfill in^this framework, providing a multisource representation: external information with visual information, internally driven information with amodal information (i.e., amodal perception makes it possible to Bcomplete^cropped objects and continues the surfaces and textures just beyond the edges of a view) or conceptual and contextual information (e.g., Intraub, 2012). Conceptual and object-to-context information is automatically activated when the visual system identifies the perceived view as a scene (i.e., as a partial view of a continuous world; e.g., Gottesman & Intraub, 2002) and generates expectations relative to the surrounding world, enabling the observer to place the perceived view in a larger spatial framework (Intraub, 2012;Intraub et al, 2015). Since visual perception is graded (with less acuity in the peripheral region), visual memory of the scene is less accurate near the boundaries of the view.…”