Tracking the Sound of Human Affection: EEG Signals Reveal Online Decoding of Socio-Emotional Expression in Human Speech and Voice

Abstract

This chapter provides a perspective from the latest EEG evidence in how brain signals enlighten the neurophysiological and neurocognitive mechanisms underlying the recognition of socioemotional expression conveyed in human speech and voice, drawing upon event-related potentials' studies (ERPs). Human sound can encode emotional meanings by diferent vocal parameters in words, real-vs. pseudo-speeches, and vocalizations. Based on the ERP indings, recent development of the three-stage model in vocal processing has highlighted initial-and late-stage processing of vocal emotional stimuli. These processes, depending on which ERP components they were mapped onto, can be divided into the acoustic analysis, relevance and motivational processing, ine-grained meaning analysis/integration/access, and higher-level social inference, as the unfolding of the time scale. ERP studies on vocal socioemotions, such as happiness, anger, fear, sadness, neutral, sincerity, conidence, and sarcasm in the human voice and speech have employed diferent experimental paradigms such as crosssplicing, crossmodality priming, oddball, stroop, etc. Moreover, task demand and listener characteristics afect the neural responses underlying the decoding processes, revealing the role of atention deployment and interpersonal sensitivity in the neural decoding of vocal emotional stimuli. Cultural orientation afects our ability to decode emotional meaning in the voice. Neurophysiological paterns were compared between normal and abnormal emotional processing in the vocal expressions, especially in schizophrenia and in congenital amusia. Future directions highlight the study on human vocal expression aligning with other nonverbal cues, such as facial and body language, and the need to synchronize listener's brain potentials with other peripheral measures.

Trends in Usage-Based and Pragmatic Language Processing and Learning: A Bibliometric Analysis on Psycholinguistics and Second-Language Acquisition Studies

“…The co-occurrence and the bibliometric coupling analysis demonstrate that researchers are most interested in the processing of various interpersonal and socio-pragmatic functions in L1 and L2 speakers in the field of psycholinguistics, as well as factors underlying L2 acquisition from children to adults, and the impact of the cognitive and neurodevelopmental impairments in the field of second-language acquisition. One important direction is to combine the real-time neurophysiological recordings (such as eye-tracking, EEGs and fMRI) with the psycholinguistic paradigms for investigating the real-world language use to reveal the neurocognitive architecture underlying pragmatic language processing and those underlying language acquisition (e.g., [6,7] for some reviews). The use of virtual reality to create naturalistic scenarios will be a good complementary approach to test the impact of real-world social interaction on one's language processing and learning [8].…”