Concurrent affective and linguistic prosody with the same emotional valence elicits a late positive ERP response

Abstract

Change in linguistic prosody generates a mismatch negativity response (MMN), indicating neural representation of linguistic prosody, while change in affective prosody generates a positive response (P3a), reflecting its motivational salience. However, the neural response to concurrent affective and linguistic prosody is unknown. The present paper investigates the integration of these two prosodic features in the brain by examining the neural response to separate and concurrent processing by electroencephalography (EEG). A spoken pair of Swedish words—[ˈfɑ́ːsɛn] phase and [ˈfɑ̀ːsɛn] damn—that differed in emotional semantics due to linguistic prosody was presented to 16 subjects in an angry and neutral affective prosody using a passive auditory oddball paradigm. Acoustically matched pseudowords—[ˈvɑ́ːsɛm] and [ˈvɑ̀ːsɛm]—were used as controls. Following the constructionist concept of emotions, accentuating the conceptualization of emotions based on language, it was hypothesized that concurrent affective and linguistic prosody with the same valence—angry [ˈfɑ̀ːsɛn] damn—would elicit a unique late EEG signature, reflecting the temporal integration of affective voice with emotional semantics of prosodic origin. In accordance, linguistic prosody elicited an MMN at 300–350 ms, and affective prosody evoked a P3a at 350–400 ms, irrespective of semantics. Beyond these responses, concurrent affective and linguistic prosody evoked a late positive component (LPC) at 820–870 ms in frontal areas, indicating the conceptualization of affective prosody based on linguistic prosody. This study provides evidence that the brain does not only distinguish between these two functions of prosody but also integrates them based on language and experience.

Perception of Prosodic Modulations of Linguistic and Paralinguistic Origin: Evidence From Early Auditory Event-Related Potentials

“… Paulmann et al (2013) indicated that although it can be differentiated as early as 200 ms (see also Paulmann and Kotz, 2008 ; Paulmann et al, 2010 ; Schirmer et al, 2013 ), affective prosody undergoes a detailed analysis to regulate the social interaction, indicated by an LPC response. In our research ( Zora et al, 2020 ), an LPC response was elicited to a match between affective prosody and emotional semantics (swear word uttered with anger in voice), reflecting semantic analysis and memory retrieval rather than simple perceptual salience.…”

“…The MMN response was even documented for the relevance of prosody in early morphological processing in Swedish, and specification of stress (lexical vs. phonological) was demonstrated to influence the processing of derivations in the brain ( Zora et al, 2019 ). These MMN results, moving beyond the signal-based perception, pinpoint that linguistic prosody is indeed accommodated in the long-term memory representations ( Honbolygó and Csépe, 2013 ; Zora et al, 2015 , 2016a , b , 2019 , 2020 ). Affective prosody was also found to modulate the amplitude of MMN response, being larger for emotional than for neutral vocalizations ( Schirmer et al, 2005 , 2007 ; Schirmer and Kotz, 2006 ; Schirmer and Escoffier, 2010 ).…”

“…Changes in linguistic prosody have for instance been shown to be perceptually more salient and therefore, eliciting a larger P3a response as compared to changes in temporal components of speech sounds ( Wang et al, 2005 ). Similarly, a larger P3a response has been indicated to affective prosody as compared to neutral prosody in vowels, pseudowords, and words ( Pakarinen et al, 2014 ; Carminati et al, 2018 ; Zora et al, 2020 ). The P3a response has also been argued to reflect not only the perceptual salience of the physical context such as affective prosody, but also all features making the stimulus contextually and motivationally more salient such as emotional semantics ( Wambacq and Jerger, 2004 ).…”

“…The MMN component is based on an oddball paradigm, where a deviant stimulus is interspersed among frequent standard stimuli, and signals the brain’s automatic reaction to deviations in auditory input at around 100–250 ms after divergence point and with a fronto-central scalp distribution ( Näätänen et al, 1978 , 2007 ). The MMN response successfully indicates neural correlates of both low-level acoustic and high-level cognitive processing associated with prosodic information (e.g., Näätänen et al, 1978 ; Honbolygó et al, 2004 , 2020 ; Weber et al, 2004 ; Friederici et al, 2007 ; Zora et al, 2015 , 2016a , b , 2019 , 2020 ; Garami et al, 2017 ). Previous research indicated MMN activation to linguistic prosody change in Swedish words that are distinguished on the sole basis of pitch accent, fasen 1 “the phase” and fasen 2 “expletive,” reflecting the activation of different lexical items in the brain based on prosody ( Zora et al, 2020 ).…”

“…The MMN response successfully indicates neural correlates of both low-level acoustic and high-level cognitive processing associated with prosodic information (e.g., Näätänen et al, 1978 ; Honbolygó et al, 2004 , 2020 ; Weber et al, 2004 ; Friederici et al, 2007 ; Zora et al, 2015 , 2016a , b , 2019 , 2020 ; Garami et al, 2017 ). Previous research indicated MMN activation to linguistic prosody change in Swedish words that are distinguished on the sole basis of pitch accent, fasen 1 “the phase” and fasen 2 “expletive,” reflecting the activation of different lexical items in the brain based on prosody ( Zora et al, 2020 ). The MMN response was even documented for the relevance of prosody in early morphological processing in Swedish, and specification of stress (lexical vs. phonological) was demonstrated to influence the processing of derivations in the brain ( Zora et al, 2019 ).…”