Cross-modal Emotional Attention: Emotional Voices Modulate Early Stages of Visual Processing

Abstract

Emotional attention, the boosting of the processing of emotionally relevant stimuli, has, up to now, mainly been investigated within a sensory modality, for instance, by using emotional pictures to modulate visual attention. In real-life environments, however, humans typically encounter simultaneous input to several different senses, such as vision and audition. As multiple signals entering different channels might originate from a common, emotionally relevant source, the prioritization of emotional stimuli should be able to operate across modalities. In this study, we explored cross-modal emotional attention. Spatially localized utterances with emotional and neutral prosody served as cues for a visually presented target in a cross-modal dot-probe task. Participants were faster to respond to targets that appeared at the spatial location of emotional compared to neutral prosody. Event-related brain potentials revealed emotional modulation of early visual target processing at the level of the P1 component, with neural sources in the striate visual cortex being more active for targets that appeared at the spatial location of emotional compared to neutral prosody. These effects were not found using synthesized control sounds matched for mean fundamental frequency and amplitude envelope. These results show that emotional attention can operate across sensory modalities by boosting early sensory stages of processing, thus facilitating the multimodal assessment of emotionally relevant stimuli in the environment.

Preferential decoding of emotion from human non-linguistic vocalizations versus speech prosody

“…In fact, vocal emotions can serve a critical role in directing visual attention to emotionally relevant NOT THE FINAL VERSION events in the environment such as faces (Brosch, Grandjean, Sander, & Scherer, 2008;Paulmann, Titone, & Pell, 2012;. In terms of composition, vocal emotion expressions are characterized by ongoing changes in several acoustic parameters that are meaningful to listeners in combination-principally, these involve differences in pitch, loudness, rhythm, and voice quality (Banse & Scherer, 1996;Juslin & Laukka, 2003).…”

MMN responses during implicit processing of changes in emotional prosody: an ERP study using Chinese pseudo-syllables

“…It is possible that at the early stage human brains are sensitive to general information of emotional prosody, rather than specific emotional category information. Previous oddball studies on emotional prosody usually interpret MMNs elicited by emotional deviants as an index of sensitivity to difference in emotional categories (Goydke et al 2004;Schirmer et al 2005), due to the evolutionary significance and the processing priority of emotional information (Brosch et al 2008(Brosch et al , 2009). Other than category processing view, our results revealed the simultaneous change detection in emotional category and physical properties, supporting the idea of early general information of emotional prosody.…”

Modulation of spatial attention to visual targets by emotional environmental sounds.

“…The interaural level differences (ILDs) of the left and right channels were adjusted to create sounds that were approximately spatially matched to the locations of the visual targets, as this offers a closer resemblance to real-world environments (where correlated auditory and visual information often arise from a single spatial location (c.f., Brosch et al 2009)), compared to simple dichotic presentation. A preliminary experiment was run with three participants to determine the ILDs which best matched the locations of the visual targets, following a similar procedure to Meienbrock, Naumer, Doehrmann, Singer, & Muckli (2007).…”

“…For example, it has been shown that emotional pictures can improve categorization of auditory cues (Tartar, de Almeida, McIntosh, Rosselli, & Nash, 2012), and that emotional pictures, compared to neutral pictures, can bias attention towards non-emotional tactile and auditory targets (Van Damme, Gallace, Spence, Crombez, & Moseley, 2009). In particular, facilitated processing of visual targets has been reported following presentation of spatially uninformative cues conveying negative (Brosch, Grandjean, Sander and Scherer, 2008a) and positive prosody (Brosch, Grandjean, Sander, & Scherer, 2008b), and this type of crossmodal exogenous spatial cueing by emotional stimuli is thought to occur at a relatively early (i.e., perceptual) stage of processing, as revealed by event-related potentials (Brosch, Grandjean, Sander, & Scherer, 2009).…”