AVLaughterCycle

Abstract

The AVLaughterCycle project aims at developing an audiovisual laughing machine, able to detect and respond to user's laughs. Laughter is an important cue to reinforce the engagement in human-computer interactions. As a first step toward this goal, we have implemented a system capable of recording the laugh of a user and responding to it with a similar laugh. The output laugh is automatically selected from an audiovisual laughter database by analyzing acoustic similarities with the input laugh. It is displayed by an Embodied Conversational Agent, animated using the audio-synchronized facial movements of the subject who originally uttered the laugh. The application is fully implemented, works in real time and a large audiovisual laughter database has been recorded as part of the project. This paper presents AVLaughterCycle, its underlying components, the freely available laughter database and the application architecture. The paper also includes evaluations of several core components of the application. Objective tests show that the similarity search engine, though simple, significantly outperforms chance for grouping laughs Portions of this work have been presented in "Proceedings of eNTERFACE'09" [36]. by speaker or type. This result can be considered as a first measurement for computing acoustic similarities between laughs. A subjective evaluation has also been conducted to measure the influence of the visual cues on the users' evaluation of similarity between laughs.

The MAHNOB Laughter database

“…There are around 300 laughs with an average duration of 1.50 s and around 1000 speech samples with an average duration of 2 s. The database is available from http://www.speech-data.jp/corpora.html. AudioVisual Laughter Cycle database (AVLC) [33]: The AVLC database was designed to elicit laughter from the participants, who were recorded while watching funny video clips for approximately 10 min. In total 24 subjects were recorded, 9 females and 15 males from various origins.…”

“…This means that the start and end point of a laughter episode is defined for the audio signal and then the corresponding video frames are extracted for all experiments conducted. In cases where it was not clear if a laughter should be considered as one episode or two consecutive episodes the video channel was used as well, similar to [33].…”

“…In the AVLC database [33] it was considered as part of the laughter whereas in [16] it was not. Therefore, we introduce two different sets of labels, one that contains the inhalation and one which does not contain it as described below.…”

“…On the other hand, it has been proposed that speech laughter should represent an independent category [58] since it is not simply laughter superimposed on articulation but a more complex vocalization. In other databases [33] and studies [62,63] it has been annotated and studied as an independent category, and we follow the same approach. Posed Smile: Used to label posed smiles.…”

“…Unfortunately, most of these data is not publicly available. To the best of our knowledge, only two publicly available audiovisual databases of elicited laughter provide suitable data for audiovisual laughter analysis, the AudioVisual Laughter Cycle database (AVLC) [33] and the MMI (part V) database [34]. The AVLC database contains recordings of 24 subjects showing 1066 laughter episodes while the MMI-part V contains recordings of 9 subjects showing 164 laughter episodes.…”