Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection

Abstract

Abstract-Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in humanrobot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task.

Facial Displays Are Tools for Social Influence

“…Active appearance models individualized for each participant enabled the development of highly discriminant feature vectors from subsegments of each video sequence, which were cross-validated and then tested. Accuracy of the system in recognizing 'success' versus 'failure' displays, despite considerable individual and intertrial variation, equaled average human recognition performance [90].…”