Perception and Automatic Recognition of Laughter from Whole-Body Motion: Continuous and Categorical Perspectives

Abstract

Despite its importance in social interactions, laughter remains little studied in affective computing. Intelligent virtual agents are often blind to users' laughter and unable to produce convincing laughter themselves. Respiratory, auditory, and facial laughter signals have been investigated but laughter-related body movements have received less attention. The aim of this study is threefold. First, to probe human laughter perception by analyzing patterns of categorisations of natural laughter animated on a minimal avatar. Results reveal that a low dimensional space can describe perception of laughter "types". Second, to investigate observers' perception of laughter (hilarious, social, awkward, fake, and non-laughter) based on animated avatars generated from natural and acted motioncapture data. Significant differences in torso and limb movements are found between animations perceived as laughter and those perceived as non-laughter. Hilarious laughter also differs from social laughter. Different body movement features were indicative of laughter in sitting and standing avatar postures. Third, to investigate automatic recognition of laughter to the same level of certainty as observers' perceptions. Results show recognition rates of the Random Forest model approach human rating levels. Classification comparisons and feature importance analyses indicate an improvement in recognition of social laughter when localized features and nonlinear models are used.

Bodily sensation maps: Exploring a new direction for detecting emotions from user self-reported data

“…Employing computers in the analysis introduces the opportunity of automatic detection of emotions, a research topic to which the literature is devoting increasing attention, proposing and evaluating classifiers based on physiological signals (Mandryk & Atkins, 2007;Fleureau et al, 2012;Wu et al, 2010;Chittaro & Sioni, 2014;Gruebler & Suzuki, 2014) or the recording of user's behavior based on cameras, e.g. facial expressions (Eleftheriadis et al, 2015;Soleymani et al, 2016), body movement (Castellano et al, 2007) and laughter from body motion (Griffin et al, 2015); microphones, e.g. speech (Deng et al, 2014) and non-verbal sound analysis (Gupta et al, 2016); or written textual communication with other users (Li & Xu, 2014) and conversational agents (Benyon et al, 2013).…”

Laughter Research: A Review of the ILHAIRE Project

“…Within ILHAIRE, another strong focus did hence lay in the investigation of laughter in the body and the perception of such cues. Within ILHAIRE, Griffin and colleagues [39] analyzed participants perception of laughter from body movements. The participants task was to categories animations of natural laughter from motion capture data replayed using faceless stick figures (characters with trunk, limbs and heads simply represented by edges).…”

“…Although some previous studies described the morphological attributes of laughter, it was still necessary within ILHAIRE to gather more detailed statistics related to specific motion patterns appearing during laughter [39]. These studies used recordings done within the project and including motion capture using high-end optical hardware.…”

“…Another proposal was made earlier [40,39] within the project where body motion was investigated for laughter type recognition among five categories (hilarious, social, awkward, fake, and non-laughter). Features characterizing hand gesture, shoulder movement, neck/spine bending, as well as kinetic energy of several upper body articulations were extracted.…”

The Belfast storytelling database: A spontaneous social interaction database with laughter focused annotation

“…However, it has taken many years for other academic domains interested in human social interaction to pay sufficient attention to this important social signal. Within the domain of Affective Computing laughter is a particularly important social signal, it signals positive affect [3] and social affiliation [4] and has important conversational functions that are likely to be crucial in creating more humanoriented interactions between computers and humans [5]. It serves as an important regulator of many functional features of human social interaction regulating topics and turn-taking within conversation, and it aids in the repair of conversations [6], [7].…”