Facial expression and prosodic prominence: Effects of modality and facial area

Swerts, Marc; Krahmer, Emiel

doi:10.1016/j.wocn.2007.05.001

Cited by 104 publications

(100 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is a question to be explored in future research. In sum, the fact that intonation is dominant confirms previous findings for Dutch (Swerts & Krahmer, 2008): The eyebrow movement cue had an effect on accent perception in Dutch, which was comparatively much smaller than the intonational effect.…”

Section: Resultssupporting

confidence: 89%

“…Finally, although intonation overrides visual cues (along the lines of Swerts & Krahmer, 2008), our findings also suggest that visual cues play a role in structural or linguistic marking (like sentence type and pragmatic meaning) as shown by the considerably lower reaction times in the AV condition comparatively to the AO condition, when running analyses over all participants.…”

Section: Resultsmentioning

confidence: 55%

“…Eyebrow movements may function as a question marker in French (Purson et al, 1999), even when they are not necessarily coordinated with fundamental frequency changes (Cavé et al, 1996). The same visual cue also has a significant effect on the perception of focus (Krahmer et al, 2002) and prominence , 2006, 2008Krahmer & Swerts, 2007) in Dutch, and may help to distinguish between specific sentence types and pragmatic meanings across languages, such as between Dutch and Catalan (Borràs-Comes & Prieto, 2011;Borràs-Comes et al, 2014;Crespo-Sendra et al, 2013). For instance, Crespo-Sendra et al (2013) have shown that Catalan participants rely more on visual cues than Dutch participants in order to perceive the contrast between neutral and focused yes-no questions, and that this is related to the kind/richness of the respective intonational cues (Catalan speakers use the same intonational contour for both pragmatic meanings, with a different pitch range, whereas Dutch speakers use different intonational contours).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The role of intonation and visual cues in the perception of sentence types: Evidence from European Portuguese varieties

Cruz

Swerts

Frota

2017

Laboratory Phonology: Journal of the Association for Laboratory Phonology

Self Cite

View full text Add to dashboard Cite

Cruz, M., et al 2017 The role of intonation and visual cues in the perception of sentence types: Evidence from European Portuguese varieties. In this paper, we explore the role of intonation and visual cues in the perception of statements and questions in two varieties of European Portuguese-the standard (SEP) and the insular variety of Azores, Ponta Delgada (PtD)-previously shown to convey sentence type contrasts by different uses of intonational means and/or facial gestures, namely eyebrow movements. Forty native speakers (20 from each variety) were exposed to SEP and PtD stimuli in a perception task with three conditions (audio only, video only, and audiovisual). The audiovisual condition includes congruent and incongruent (both original and manipulated) stimuli, where there is either a match or a mismatch between the auditory and visual features as potential cues for a specific sentence type. We concluded that both SEP and PtD participants rely more on intonation than on eyebrow movement to identify sentence types, even when exposed to incongruent audiovisual stimuli. In the absence of audio information, unexpectedly, participants do not interpret eyebrow raising as a question marker, not even when perceiving stimuli from their native variety. When exposed to non-native audiovisual stimuli, both SEP and PtD participants present longer reaction times (RTs), especially for incongruent stimuli. Finally, although we confirm the strength of intonation over visual cues, RTs in the audiovisual condition are significantly shorter than in the audio condition, thus pointing to the relevance of visual cues for structural/linguistic marking.

show abstract

Section: Resultssupporting

confidence: 89%

Section: Resultsmentioning

confidence: 55%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The role of intonation and visual cues in the perception of sentence types: Evidence from European Portuguese varieties

Cruz

Swerts

Frota

2017

Laboratory Phonology: Journal of the Association for Laboratory Phonology

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is linked to the syntax of an utterance, the attitudes of the speaker and the emotions (Scherer 1988). Facial expressions can serve as cues to signal acoustic prominence (Bolinger 1989;Beskow et al 2006;Swerts & Krahmer 2008). Indeed, detailed visual-acoustic analyses have shown the relation between pitch accents and facial signals.…”

Section: Intonational Markersmentioning

confidence: 99%

“…More facial actions occur on a stressed word. Indeed, Humans use facial actions as a cue of accent signalling (Krahmer & Swerts 2004;Swerts & Krahmer 2008). Perceptual tests have been conducted to understand how facial actions participate in the perception of prominence.…”

Section: Intonational Markersmentioning

confidence: 99%

Modelling multimodal expression of emotion in a virtual agent

Pélachaud

2009

Phil. Trans. R. Soc. B

111

View full text Add to dashboard Cite

Over the past few years we have been developing an expressive embodied conversational agent system. In particular, we have developed a model of multimodal behaviours that includes dynamism and complex facial expressions. The first feature refers to the qualitative execution of behaviours. Our model is based on perceptual studies and encompasses several parameters that modulate multimodal behaviours. The second feature, the model of complex expressions, follows a componential approach where a new expression is obtained by combining facial areas of other expressions. Lately we have been working on adding temporal dynamism to expressions. So far they have been designed statically, typically at their apex. Only full-blown expressions could be modelled. To overcome this limitation, we have defined a representation scheme that describes the temporal evolution of the expression of an emotion. It is no longer represented by a static definition but by a temporally ordered sequence of multimodal signals.

show abstract

Visual Contribution to Word Prominence Detection in a Playful Interaction Setting

Heckmann

2013

Natural Interaction With Robots, Knowbots and Smartphones

View full text Add to dashboard Cite

Facial expression and prosodic prominence: Effects of modality and facial area

Cited by 104 publications

References 40 publications

The role of intonation and visual cues in the perception of sentence types: Evidence from European Portuguese varieties

The role of intonation and visual cues in the perception of sentence types: Evidence from European Portuguese varieties

Modelling multimodal expression of emotion in a virtual agent

Visual Contribution to Word Prominence Detection in a Playful Interaction Setting

Contact Info

Product

Resources

About