Gesture synthesis adapted to speech emphasis

Fernández-Baena, Adso; Montaño, Raúl; Antonijoan, Marc; Roversi, Arturo; Miralles, David; Álías, Francesc

doi:10.1016/j.specom.2013.06.005

Cited by 18 publications

(13 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Second, we found that the position of the peak of prominence in the gesture (the gesture apex) is sensitive not only to the position of the accented syllable (which had been found in many previous studies; Fernandez-Baena et al, 2014;Graf et al, 2002;Hadar et al, 1983;Ishi et al, 2014), but that it is also highly sensitive to the distance to the upcoming intonational phrase boundary. The position of the accented syllable within the prosodic word determined where the gesture apex will be produced (because gesture apexes tend to occur within its limits).…”

Section: Discussionsupporting

confidence: 63%

“…These studies take the position of the accented syllables as the key prosodic landmark with which gesture movements align, but they do not take into account intonational phrase boundaries. In general, they found a similar temporal alignment pattern as had been shown for hand gestures: accented syllables are the anchoring point in speech for the most prominent part of a head movement, the gesture apex (defined as the specific point in time when the head changes its direction in the vertical or lateral movement) (Alexanderson et al, 2013;Ambrazaitis et al, 2015;Fernandez-Baena et al, 2014;Goldenberg et al, 2014;Graf et al, 2002;Hadar et al, 1983;Ishi et al, 2014;Kim et al, 2014). However, these studies also reported variability in this alignment pattern.…”

Section: Introductionsupporting

confidence: 58%

“…Previous research on the alignment of head gestures with speech had shown that accented syllables were the anchoring site for head apexes (Alexanderson et al, 2013;Fernandez-Baena et al, 2014;Goldenberg et al, 2014;Graf et al, 2002;Hadar et al, 1983;Ishi et al, 2014). Yet, they also reported variability in this pattern.…”

Section: General Discussion and Conclusionmentioning

confidence: 99%

See 2 more Smart Citations

The timing of head movements: The role of prosodic heads and edges

Esteve-Gibert

Borràs-Comes

Asor

et al. 2017

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

This study examines the influence of the position of prosodic heads (accented syllables) and prosodic edges (prosodic word and intonational phrase boundaries) on the timing of head movements. Gesture movements and prosodic events tend to be temporally aligned in the discourse, the most prominent part of gestures typically being aligned with prosodically prominent syllables in speech. However, little is known about the impact of the position of intonational phrase boundaries on gesture-speech alignment patterns. Twenty-four Catalan speakers produced spontaneous (experiment 1) and semi-spontaneous head gestures with a confirmatory function (experiment 2), along with phrasefinal focused words in different prosodic conditions (stress-initial, stress-medial, and stress-final). Results showed (a) that the scope of head movements is the associated focused prosodic word, (b) that the left edge of the focused prosodic word determines where the interval of gesture prominence starts, and (c) that the speech-anchoring site for the gesture peak (or apex) depends both on the location of the accented syllable and the distance to the upcoming intonational phrase boundary. These results demonstrate that prosodic heads and edges have an impact on the timing of head movements, and therefore that prosodic structure plays a central role in the timing of co-speech gestures.

show abstract

Section: Discussionsupporting

confidence: 63%

Section: Introductionsupporting

confidence: 58%

Section: General Discussion and Conclusionmentioning

confidence: 99%

See 1 more Smart Citation

The timing of head movements: The role of prosodic heads and edges

Esteve-Gibert

Borràs-Comes

Asor

et al. 2017

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

show abstract

“…In the Database Annotation approach, a database is annotated with gestures, constructing a gesture dictionary, together with a set of gestural dimensions such as amplitude, fluidity, power and speed, allowing to increase the expressiveness of the gestures [32,23,19]. Finally, Expert Systems are used when no corpus is available, by recording an ad hoc corpus to obtain animations and their corresponding video recordings, together with the level of intensity and strength of gestures, lexemes and gesture phases denoting the start, main stroke and retraction of gestures over time, or with different bags of words to enable gesture primitives selection [12,29,25].…”

Section: Prosody and Linguistic Information For Virtual Agent Gesturementioning

confidence: 99%

Part-of-Speech and Prosody-based Approaches for Robot Speech and Gesture Synchronization

Pérez-Mayos¹,

Farrús²,

Segura³

2019

J Intell Robot Syst

View full text Add to dashboard Cite

Humanoid robots are already among us and they are beginning to assume more social and personal roles, like guiding and assisting people. Thus, they should interact in a human-friendly manner, using not only verbal cues but also synchronized non-verbal and para-verbal cues. However, available robots are not able to communicate in this multimodal way, being just able to perform predefined gesture sequences, handcrafted to accompany specific utterances. In the current paper, we propose a model based on three different approaches to extend humanoid robots communication behaviour with upper body gestures synchronized with the speech for novel utterances, exploiting part-of-speech grammatical information, prosody cues, and a combination of both. User studies confirm that our methods are able to produce natural, appropriate and good timed gesture sequences synchronized with speech, using both beat and emblematic gestures.

show abstract

“…Dans une perspective de communication, parole et mouvements forment un ensemble indissociable : longtemps considérée comme un module complémentaire, la gestualité est désormais comprise et analysée comme un élément constitutif de la langue orale (McNeill, 1985 ;. L'expression emphatique peut donc comporter des marques gestuelles qui contribuent au sens emphatique de l'énoncé et qui parfois l'annoncent : une étude récente (Fernández-Baena et al, 2014) montre que le geste précède souvent la tournure emphatique. McNeill (2005), les gestes comportent quatre dimensions qui peuvent être combinées : iconiques, métaphoriques, déictiques et rythmiques.…”

Section: Une Difficulté : L'expression De L'emphaseunclassified