Multisensory Interactions in the Real World

Abstract

The interactions between the senses are essential for cognitive functions such as perception, attention, and action planning. Past research helped understanding of multisensory processes in the laboratory. Yet, the efforts to extrapolate these findings to the real-world are scarce. Extrapolation to real-world contexts is important for practical and theoretical reasons. Multisensory phenomena might be expressed differently in real-world settings compared to simpler laboratory situations. Some effects might become stronger, others may disappear, and new outcomes could be discovered. This Element discusses research that uncovers multisensory interactions under complex environments, with an emphasis on the interplay of multisensory mechanisms with other processes.

Not so automatic: Task relevance and perceptual load modulate cross-modal semantic congruence effects on spatial orienting

“…Recently, the importance of studying multisensory interactions in realistic environments has been highlighted (e.g. Soto-Faraco et al, 2019;Matusz et al, 2019). One particularly relevant point refers to the interaction between these multisensory processes and attention, given that in realistic contexts, perceptual load tends to be high, compared to the idealised conditions of the current (and previous experiments).…”

From cognitive control to visual incongruity: Conflict detection in surrealistic images

“…In the spirit of the ecological validity underlying this study, we attempted to test our hypotheses under close to realistic conditions to improve the feasibility of applying such measurements outside the laboratory [23,24]. Therefore, we used a wearable EEG headset that makes it more agile and applicable to real-world testing.…”

Book Review

“…Notice how, according to such a themed (not to mention visually dominant; see Hutmacher, 2019) unisensory approach, object recognition is treated as essentially a visual problem whereas speech perception, naturally enough, is seen as belonging squarely in the auditory section (which is where it also appears in Moore's book). According to such organizational frameworks, there is no real recognition that vision, specifically lip movements, or even gesture, might play a key role in communication, as discussed in Soto-Faraco et al's (2019), recently published volume, Multisensory Interactions in the Real World. I have just reviewed the latter book for this very journal (see Spence, in press).…”

“…One of the other possible blindspots is any consideration of perceptual load (e.g., Forster and Spence, 2018;Lavie, 1995Lavie, , 2005Murphy et al, 2017a, b;Rees et al, 2001). It can be argued that resolution of the question of when attention modulates multisensory integration can only really be resolved with reference to the load-based account of attentional selection (see also Soto-Faraco et al, 2019, on this theme).…”