Annotation of spoken syntax in relation to prosody and multimodal pragmatics

Hunyadi, László; Szekrényes, István; Borbély, Anna; Kiss, Hermina

doi:10.1109/coginfocom.2012.6422039

Cited by 8 publications

(1 citation statement)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reviews of existing corpora that can be reused for studying multimodal communication can be found in [73] and [74]. A corpus that was elaborated on a similar domain as ours is the HuComTech corpus [75], which contains recordings of role-played job interviews. However the subjects of the corpus were acting as recruiters and were not professional recruiters, and their behaviors may not have reflected behaviors of actual recruiters.…”

Section: Multimodal Corpus Acquisition and Annotationmentioning

confidence: 99%

A Methodology for the Automatic Extraction and Generation of Non-Verbal Signals Sequences Conveying Interpersonal Attitudes

Chollet

Ochs

Pélachaud

2019

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

Depending on their application, Embodied Conversational Agents (ECAs) must be able to express various affects or social constructs such as emotions or social attitudes. Non-verbal signals, such as smiles or gestures, contribute to the expression of attitudes. Social attitudes affect the whole behavior of a person: as Scherer puts it, they are "characteristic of an affective style that colors the entire interaction" [1]. Moreover, recent findings have demonstrated that non-verbal signals are not interpretated in isolation but along with surrounding signals: for instance, a smile followed by a gaze aversion and a head aversion may signal embarassment rather than amusement [2]. Non-verbal behavior planning models designed to allow ECAs to express attitudes should thus consider complete sequences of non-verbal signals and not only signals independently of one another. However, existing models do not take this into account, or in a limited manner. The contribution of this paper is a methodology for the automatic extraction of sequences of non-verbal signals characteristic of a social phenomenon from a multimodal corpus, and a non-verbal behavior planning model that takes into account sequences of non-verbal signals rather than signals independently. This methodology is applied to design a virtual recruiter capable of expressing social attitudes, which is then evaluated in and out of an interaction context.

show abstract

Section: Multimodal Corpus Acquisition and Annotationmentioning

confidence: 99%

A Methodology for the Automatic Extraction and Generation of Non-Verbal Signals Sequences Conveying Interpersonal Attitudes

Chollet

Ochs

Pélachaud

2019

IEEE Trans. Affective Comput.

View full text Add to dashboard Cite

show abstract

Incompleteness and Fragmentation: Possible Formal Cues to Cognitive Processes Behind Spoken Utterances

Hunyadi

Kiss

Szekrényes

2015

Intelligent Decision Technology Support in Practice

View full text Add to dashboard Cite

Pragmatic Multimodality: Effects of Nonverbal Cues of Focus and Certainty in a Virtual Human

Freigang

Klett

Kopp

2017

Intelligent Virtual Agents

View full text Add to dashboard Cite

Story for stimulus video Controlled information structure Theme marked as: xxx Rheme/comment marked as: xxx Focus/contrast in rheme/comment marked as: xxx (gesture stroke is placed here in F+/F-/E-) Focus/contrast in theme marke das: xxx 1 Ich bin also Billie, der virtuelle Agent. 2 Virtuelle Agenten werden zum Beispiel in Assistenzsystemen eingesetzt. 3 Mit Hilfe dieser Systeme können Menschen barrierefrei mit Computern oder Maschinen kommunizieren. 4 Der technische Begriff für diesen barrierefreien Kontakt ist die Mensch-Maschine-Schnittstelle. 5 Als Assistenzsystem werde ich am CITEC eingesetzt. 6 Das CITEC ist ein Exzellenzcluster der Deutschen Forschungsgemeinschaft. 7 Die ersten drei Buchstaben in CITEC stehen für den Titel "Kognitive Interaktionstechnologie". 8 Gleichzeitig ist es auch ein Forschungsbau der Universität Bielefeld. 9 Dieser ist auf dem neuen Campus Nord der Uni angesiedelt. 10 Als Fachausdruck für virtuelle Agenten wie mich verwenden einige Forscher den Begriff "verkörperte konversationale Agenten". 11 Ein anderer Ausdruck für uns ist auch "virtuelle Menschen". 12 Wir sind Charaktere, die in einer virtuellen Welt grafisch repräsentiert werden. 13 Uns liegen umfangreiche kognitive Komponenten zugrunde.

show abstract

Annotation of spoken syntax in relation to prosody and multimodal pragmatics

Cited by 8 publications

References 3 publications

A Methodology for the Automatic Extraction and Generation of Non-Verbal Signals Sequences Conveying Interpersonal Attitudes

A Methodology for the Automatic Extraction and Generation of Non-Verbal Signals Sequences Conveying Interpersonal Attitudes

Incompleteness and Fragmentation: Possible Formal Cues to Cognitive Processes Behind Spoken Utterances

Pragmatic Multimodality: Effects of Nonverbal Cues of Focus and Certainty in a Virtual Human

Contact Info

Product

Resources

About