The role of multisensory memories in unisensory object discrimination

Lehmann, Stefanie; Murray, Micah M.

doi:10.1016/j.cogbrainres.2005.02.005

Cited by 165 publications

(165 citation statements)

References 31 publications

Supporting

Mentioning

145

Contrasting

Unclassified

Order By: Relevance

“…This would suggest the existence of true multisensory memory traces. Consistent with this notion, Lehmann and Murray (2005) showed that memory for object drawings was improved if the initial encounter with the drawing was accompanied by a congruent sound (relative to no sound at all) but not impaired if the initial encounter was accompanied by an incongruent sound. They suggested that this was attributable to common activation of singular multisensory representations during encoding in the congruent case and parallel activation of distinct representations (and thus no interference) in the incongruent case.…”

Section: Neurophysiological Indices Of Cross-modal Congruitysupporting

confidence: 69%

What You See Is Not (Always) What You Hear: Induced Gamma Band Responses Reflect Cross-Modal Interactions in Familiar Object Recognition

Yuval-Greenberg

Deouell²

2007

J. Neurosci.

111

View full text Add to dashboard Cite

Gamma-band responses (GBRs) are hypothesized to reflect neuronal synchronous activity related to activation of object representations. However, it is not known whether synchrony in the gamma range is also related to multisensory object processing. We investigated the effect of semantic congruity between auditory and visual information on the human GBR. The paradigm consisted of a simultaneous presentation of pictures and vocalizations of animals, which were either congruent or incongruent. EEG was measured in 17 students while they attended either the auditory or the visual stimulus and performed a recognition task. Behavioral results showed a congruity effect, indicating that information from the unattended modality affected behavior. Irrelevant visual information affected auditory recognition more than irrelevant auditory information affected visual recognition, suggesting a bias toward reliance on visual information in object recognition. Whereas the evoked (phase-locked) GBR was unaffected by congruity, the induced (non-phase-locked) GBR was increased for congruent compared with incongruent stimuli. This effect was independent of the attended modality. The results show that integration of information across modalities, based on semantic congruity, is associated with enhanced synchronized oscillations at the gamma band. This suggests that gamma-band oscillations are related not only to low-level unimodal integration but also to the formation of object representations at conceptual multisensory levels.

show abstract

Section: Neurophysiological Indices Of Cross-modal Congruitysupporting

confidence: 69%

What You See Is Not (Always) What You Hear: Induced Gamma Band Responses Reflect Cross-Modal Interactions in Familiar Object Recognition

Yuval-Greenberg

Deouell²

2007

J. Neurosci.

111

View full text Add to dashboard Cite

show abstract

“…Stein and Meredith, 1993). Such multisensory processing can affect a range of different behavioral parameters, such as reaction times (Welch and Warren, 1986;Raab, 1962), stimulus detection rate (Grant and Seitz, 2000), accuracy of stimulus identification (Giard and Peronnet, 1999) as well as learning effects on stimulus processing Lehmann and Murray, 2005). While for example the decrease in reaction times under multisensory conditions have been largely reported for human subjects in auditory-visual recognition tasks, no behavioral data to our knowledge are available in monkeys performing similar protocols.…”

Section: Role Of Non-specific Connections?mentioning

confidence: 99%

Multisensory anatomical pathways

2009

View full text Add to dashboard Cite

In order to interact with the multisensory world that surrounds us, we must integrate various sources of sensory information (vision, hearing, touch. . .). A fundamental question is thus how the brain integrates the separate elements of an object defined by several sensory components to form a unified percept. The superior colliculus was the main model for studying multisensory integration. At the cortical level, until recently, multisensory integration appeared to be a characteristic attributed to high-level association regions. First, we describe recently observed direct cortico-cortical connections between different sensory cortical areas in the non-human primate and discuss the potential role of these connections. Then, we show that the projections between different sensory and motor cortical areas and the thalamus enabled us to highlight the existence of thalamic nuclei that, by their connections, may represent an alternative pathway for information transfer between different sensory and/or motor cortical areas. The thalamus is in position to allow a faster transfer and even an integration of information across modalities. Finally, we discuss the role of these non-specific connections regarding behavioral evidence in the monkey and recent electrophysiological evidence in the primary cortical sensory areas.

show abstract

“…Further, semantic links were established in Experiment 3 between initially unrelated visual and auditory cues in the learning phase; Hearing the newly associated auditory cues then improved performance in a subsequent visual search task. Thus, while the learning of initially unrelated audiovisual cues does not increase subsequent unisensory recognition (Lehmann and Murray 2005;Murray 2012, 2013;Thelen et al 2015), it does seem to increase multisensory integration (i.e., visual search performance was increased when the learned sound was present in the test phase).…”

Section: Implications and Contributionsmentioning

confidence: 99%

Multisensory brand search: How the meaning of sounds guides consumers’ visual attention.

Knoeferle

Velasco

et al. 2016

Journal of Experimental Psychology: Applied

View full text Add to dashboard Cite

Building on models of crossmodal attention, the present research proposes that brand search is inherently multisensory, in that the consumers' visual search for a specific brand can be facilitated by semantically related stimuli that are presented in another sensory modality. A series of five experiments demonstrates that the presentation of spatially non-predictive auditory stimuli associated with products (e.g., usage sounds or product-related jingles) can crossmodally facilitate consumers' visual search for, and selection of, products. Eye-tracking data (Experiment 2) revealed that the crossmodal effect of auditory cues on visual search manifested itself not only in reaction times, but also in the earliest stages of visual attentional processing, thus suggesting that the semantic information embedded within sounds can modulate the perceptual saliency of the target products' visual representations. Crossmodal facilitation was even observed for newly-learnt associations between unfamiliar brands and sonic logos, implicating multisensory short-term learning in establishing audiovisual semantic associations. The facilitation effect was stronger when searching complex rather than simple visual displays, thus suggesting a modulatory role of perceptual load. : visual attention; product search; crossmodal; sound; learning MULTISENSORY BRAND SEARCH 4 Multisensory Brand Search: KeywordsHow the Meaning of Sounds Guides Consumers' Visual AttentionSearching for a product, no matter whether in a traditional retail environment or in an online store, can be challenging. A typical shopper may be exposed to as many as 1,000 different products and brands per minute whilst strolling down the aisles of the typical supermarket (Robinson 1999). In light of this overwhelming product choice, reducing the visual search time, and facilitating the customer's search for specific products (or product categories) is of great interest to marketers (see Wedel and Pieters 2007, for a review detailing the importance of visual attention in marketing).One way in which this could be achieved is by engaging/stimulating the customer's other senses. However, these studies have typically used "basic", decontextualized stimuli (e.g., rotated line segments and Gabor patches) and require hundreds of within-participant trials to obtain relatively small effects. It thus remains an open question as to whether multisensory stimulation substantially affects consumers' search behaviour in real-life settings and in the more complex human-made environments of consumer search. Indeed, in the consumer behaviour literature, the potential effects of crossmodal stimulation on visual processing have largely been neglected (but see Hagtvedt and Brasel, in press). In particular, studies examining consumers' brand search have focused on the visual modality (e.g., Atalay, Bodur, and Rasolofoarison 2012;Pieters and Warlop 1999;Van der Lans, Pieters, and Wedel 2008). MULTISENSORY BRAND SEARCH 5To address this gap in the literature, the present paper examine...

show abstract

The role of multisensory memories in unisensory object discrimination

Cited by 165 publications

References 31 publications

What You See Is Not (Always) What You Hear: Induced Gamma Band Responses Reflect Cross-Modal Interactions in Familiar Object Recognition

What You See Is Not (Always) What You Hear: Induced Gamma Band Responses Reflect Cross-Modal Interactions in Familiar Object Recognition

Multisensory anatomical pathways

Multisensory brand search: How the meaning of sounds guides consumers’ visual attention.

Contact Info

Product

Resources

About