The Belfast storytelling database: A spontaneous social interaction database with laughter focused annotation

McKeown, Gary; Curran, William J.; Wagner, Johannes; Lingenfelser, Florian; André, Elisabeth

doi:10.1109/acii.2015.7344567

Cited by 16 publications

(16 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While a major part of research in social signal processing and affective computing focuses on the analysis of dyadic interactions [8,33,38], a growing body of work on multi-person interactions has developed in recent years. Among the social concepts that have been studied in multi-person interactions from a computational perspective are turn-taking [6,30,39], laughter [32], general interest level of the group [18], and engagement of individuals inside the group [36]. Cohesion is one of the more abstract concepts and describes the tendency of group members to create social bonds and stay united as a group.…”

Section: Automatic Analysis Of Multi-person Interactionsmentioning

confidence: 99%

Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour

Müller

Huang

Bulling

2018

23rd International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

Rapport, the close and harmonious relationship in which interaction partners are "in sync" with each other, was shown to result in smoother social interactions, improved collaboration, and improved interpersonal outcomes. In this work, we are first to investigate automatic prediction of low rapport during natural interactions within small groups. This task is challenging given that rapport only manifests in subtle non-verbal signals that are, in addition, subject to influences of group dynamics as well as inter-personal idiosyncrasies. We record videos of unscripted discussions of three to four people using a multi-view camera system and microphones. We analyse a rich set of non-verbal signals for rapport detection, namely facial expressions, hand motion, gaze, speaker turns, and speech prosody. Using facial features, we can detect low rapport with an average precision of 0.7 (chance level at 0.25), while incorporating prior knowledge of participants' personalities can even achieve early prediction without a drop in performance. We further provide a detailed analysis of different feature sets and the amount of information contained in different temporal segments of the interactions.

show abstract

Section: Automatic Analysis Of Multi-person Interactionsmentioning

confidence: 99%

Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour

Müller

Huang

Bulling

2018

23rd International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

show abstract

“…Modality Type Participants Duration Laughter Instances Emotion Annotation AVLC [56] A, V, FT Induced, Posed 24 64m 28s 1066 No BELFAST * [29] A, V, K Interactive 21 106m 2336 Yes BINED * [48] A, V Induced 256 29m 45s 289 Yes MAHNOB [36] A, V, P Induced, Posed 22 18m 59s 563 No MMLI [32] A, V, P, FT, BM Induced, Interactive 16 31m 439 No RECOLA [43] A, V, P Interactive 46 17m 58s 974 Yes SEMAINE * [30] A, V Interactive 150 2m 05s 443 Yes psychotherapy, where a computer can monitor the reactions of the patients through multimodal signals [22]. Moreover, contextual information on affect is necessary to understand the interplay between the kind of laughter and emotion in the enactment.…”

Section: Databasementioning

confidence: 99%

Automatic Recognition of Affective Laughter in Spontaneous Dyadic Interactions from Audiovisual Signals

Kantharaju

Ringeval

Besacier

2018

Proceedings of the 20th ACM International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Laughter is a highly spontaneous behavior that frequently occurs during social interactions. It serves as an expressive-communicative social signal which conveys a large spectrum of affect display. Even though many studies have been performed on the automatic recognition of laughter -or emotion -from audiovisual signals, very little is known about the automatic recognition of emotion conveyed by laughter. In this contribution, we provide insights on emotional laughter by extensive evaluations carried out on a corpus of dyadic spontaneous interactions, annotated with dimensional labels of emotion (arousal and valence). We evaluate, by automatic recognition experiments and correlation based analysis, how different categories of laughter, such as unvoiced laughter, voiced laughter, speech laughter, and speech (non-laughter) can be differentiated from audiovisual features, and to which extent they might convey different emotions. Results show that voiced laughter performed best in the automatic recognition of arousal and valence for both audio and visual features. The context of production is further analysed and results show that, acted and spontaneous expressions of laughter produced by a same person can be differentiated from audiovisual signals, and multilingual induced expressions can be differentiated from those produced during interactions.

show abstract

“…In recent years, a large number of multimodal datasets on human interaction have been collected and published [29,38,45,46]. We brie y review some by dividing them into two groups, according to the number of interlocutors.…”

Section: Related Workmentioning

confidence: 99%

“…In these datasets, the number of interlocutors is not constrained. Popular examples of such datasets are the Belfast storytelling dataset [29], which collects spontaneous social interactions with laughter, the Multimodal multiperson corpus of laughter in interaction [34] which collects multimodal data of laughter with focus on full body movements and di erent laughter 1 h ps://sewaproject.eu Figure 1: Sketch of the recording setup: Novice (le ) and expert (right) are located in separate rooms while having a screen-mediated conversation. e interaction is monitored from a third room (middle) and recorded in sync.…”

Section: Related Workmentioning

confidence: 99%

“…We opted for a screen-mediated recording for a twofold purpose, rst it allowed us to record a face-to-face conversation without the need of multiple cameras recording from di erent angles as in classical face-to-face se ings of other corpora (e.g. [29,34]). Secondly, this setup is closer to a scenario where a virtual agent is displayed on a screen.…”

Section: Design Principlesmentioning

confidence: 99%

See 1 more Smart Citation

The NoXi database: multimodal recordings of mediated novice-expert interactions

Cafaro

Wagner

Baur

et al. 2017

Proceedings of the 19th ACM International Conference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

We present a novel multi-lingual database of natural dyadic noviceexpert interactions, named NoXi, featuring screen-mediated dyadic human interactions in the context of information exchange and retrieval. NoXi is designed to provide spontaneous interactions with emphasis on adaptive behaviors and unexpected situations (e.g. conversational interruptions). A rich set of audio-visual data, as well as continuous and discrete annotations are publicly available through a web interface. Descriptors include low level social signals (e.g. gestures, smiles), functional descriptors (e.g. turn-taking, dialogue acts) and interaction descriptors (e.g. engagement, interest, and uidity). CCS CONCEPTS•Information systems → Database design and models; Semistructured data; Data streams; •Human-centered computing → Systems and tools for interaction design; KEYWORDS A ective computing, multimodal corpora, multimedia databases ACM Reference format:

show abstract

The Belfast storytelling database: A spontaneous social interaction database with laughter focused annotation

Cited by 16 publications

References 12 publications

Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour

Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour

Automatic Recognition of Affective Laughter in Spontaneous Dyadic Interactions from Audiovisual Signals

The NoXi database: multimodal recordings of mediated novice-expert interactions

Contact Info

Product

Resources

About