Relationships among Facial, Prosodic, and Lexical Channels of Emotional Perceptual Processing

Abstract

This study was designed to address the issue of whether there is a general processor for the perception of emotion or whether there are separate processors. We examined the relationships among three channels of emotional communication in 100 healthy right-handed adult males and females. The channels were facial, prosodic/intonational, and lexical/verbal; both identi® cation and discrimination tasks of emotional perception were utilised. Statistical analyses controlled for nonemotional perceptual factors and subject characteristics (i.e. demographic and general cognitive). For identi® cation, multiple signi® cant correlations were found among the channels. For discrimination, fewer correlations were signi® cant. Overall, these results provide support for the notion of a general processor for emotional perceptual identi® cation in normal adult subjects.

Integration of Face and Voice During Emotion Perception: Is There Anything Gained for the Perceptual System Beyond Stimulus Modality Redundancy?

“…But in the absence of any limitations, multisensory perception would be rather inefficient to deal with these natural variations. The new behavioral results presented in this chapter are in line with this assumption and previous studies (see , as they show that the combination of an emotional facial expression with an affective tone of voice does not reflect a general (amodal) perceptual effect (see Massaro, 1998;Massaro & Egan, 1996;Borod et al, 2000), but instead it may serve a specific optimization function for the organism, aimed at binding selectively visual and non-lexical (psycho-acoustic) auditory cues during emotion perception, as these two cues usually convey simultaneously and naturistically critical and converging emotional information about the actual mental state, and possible intentions or action tendencies of peers or conspecifics (Frijda, 1989). For this reason, is the combination of multisensory inputs during emotion perception probably a key perceptual process relying on specific cognitive processes and neural systems (see Pourtois et al, 2005), likely sharing similarities with other multisensory objects, including space perception (see Bertelson, 1999), even though this conjecture remains largely speculative at this stage and it would need some direct confirmation at the empirical level as well.…”

“…Indeed, multisensory perception of emotion consists of enhancing detection and discrimination of emotions, as well as speed responsiveness to these 15 highly relevant biological stimuli (Sander, Grafman, & Zalla, 2003). The fact that the perception of emotions is by nature multimodal and audio-visual, allows the perceptual system to disambiguate the actual functional meaning of the emotional input using a stable amodal or supramodal representation (see Farah, Wong, Monheit, & Morrow, 1989;Borod et al, 2000). There are large inter-individual differences between human beings (as well as animals) in the ability to express and perceive different emotions.…”

“…Likewise, studies looking at correlations between emotion face, affective voice and emotion word perception do exist and have been performed in the past (e.g. Borod et al, 1998;Borod et al, 2000), but these valuable studies suggesting the likely existence of amodal perceptual mechanisms during emotion perception do not directly address the question of how multiple sensory cues (e.g. face and voice) may be integrated during emotion perception, and what the resulting emotion percept may be.…”

“…Two extremes, but equally plausible alternatives, may be suggested in this respect. Either the crossmodal bias between the voice and the face during emotion perception actually reflects the existence of a general mechanism for affect perception whereby the perceptual system (blindly) samples and merges all sources of affect information available at a given moment (time) and position (space) (Borod et al, 2000). Or alternatively, the crossmodal bias effect during emotion perception is narrowly restricted to the combination of an emotional facial expression with an effective tone of voice, and this specific audio-visual situation requires selective (perceptual) mechanisms (see Pourtois et al, 2005).…”

Decoding speech prosody: Do music lessons help?

“…Indeed, there may be a general processor for the identification of emotional stimuli that extends to visual stimuli. For example, performance is correlated across tasks when adults are required to make judgments about the emotional content of speech prosody in one task, and the emotions expressed by faces in another task (Borod et al, 2000).…”