A 3-D Audio-Visual Corpus of Affective Communication

Abstract

Communication between humans deeply relies on the capability of expressing and recognizing feelings. For this reason, research on human-machine interaction needs to focus on the recognition and simulation of emotional states, prerequisite of which is the collection of affective corpora. Currently available datasets still represent a bottleneck for the difficulties arising during the acquisition and labeling of affective data. In this work, we present a new audio-visual corpus for possibly the two most important modalities used by humans to communicate their emotional states, namely speech and facial expression in the form of dense dynamic 3-D face geometries. We acquire high-quality data by working in a controlled environment and resort to video clips to induce affective states. The annotation of the speech signal includes: transcription of the corpus text into the phonological representation, accurate phone segmentation, fundamental frequency extraction, and signal intensity estimation of the speech signals. We employ a real-time 3-D scanner to acquire dense dynamic facial geometries and track the faces throughout the sequences, achieving full spatial and temporal correspondences. The corpus is a valuable tool for applications like affective visual speech synthesis or view-independent facial expression recognition.

Random Forests for Real Time 3D Face Analysis

“…We further performed 5-fold, subject-independent cross validations on the B3D(AC) 2 database [24], and the BU 3DF E database rendered both in frontal pose and with random rotations added. Table 4 shows mean and standard deviation of the errors in millimeters for all the analyzed facial features.…”

“…As a first dataset, we chose B3D(AC) 2 [24]. This is a relatively naturalistic and large set of high quality, dynamic facial scans, with subjects recorded using the 3D scanner of [66] while pronouncing a set of 40 predefined sentences both in a neutral and in an induced emotional state.…”

Expressive Modulation of Neutral Visual Speech

“…Some of our experiments use the Biwi 3D Audiovisual Corpus of Affective Communication, Fanelli et al [2010a]. This re-acted corpus comprises of depth scanned data captured using a novel method allowing for 3D data capture using relatively inexpensive equipment (a digital projector and three cameras), by employing phase shifting and stereo unwrapping (see Weise et al [2007], for a detailed description).…”

Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data

“…Experiments were conducted on the BIWI 3D Audiovisual Corpus of Affective Communication [40] comprising a total of 1109 sentences (4.67 seconds long on average) uttered by 14 native English speakers (6 males and 8 females). The dense dynamic face scans were acquired at 25 frames per second by a realtime 3D scanner and the voice signal was captured by a professional microphone at a sampling rate of 16kHz.…”

“…In the offline version, the 3D BIWI dataset [40] is used to train the GMM model between any two speakers. The training is done on 40 utterances performed in a neutral manner by both speakers.…”