Semantic 3D motion retargeting for facial animation

Abstract

We present a system for realistic facial animation that decomposes facial Motion Capture data into semantically meaningful motion channels based on the Facial Action Coding System. A captured performance is retargeted onto a morphable 3D face model based on a semantically corresponding set of 3D scans. The resulting facial animation reveals a high level of realism by combining the high spatial resolution of a 3D scanner with the high temporal accuracy of motion capture data that accounts for subtle facial movements with sparse measurements. Such an animation system allows us to systematically investigate human perception of moving faces. It offers control over many aspects of the appearance of a dynamic face, while utilizing as much measured data as possible to avoid artistic biases. Using our animation system, we report results of an experiment that investigates the perceived naturalness of facial motion in a preference task. For expressions with small amounts of headmotion, we find a benefit for our part-based generative animation system that is capable of local animation over an example-based approach that deforms the whole face at once

Categorizing identity from facial motion

“…Each unit is then plotted as a time course so that the spatiotemporal properties of local movements can be represented. This technique has been applied to motion--capture data to create highly controlled and meaningful facial animations (e.g., Curio et al, 2006;Dobs et al, 2014). The advantage here is that facial motion is annotated accurately and precisely with reference to underlying muscle activations.…”

“…The advantage here is that facial motion is annotated accurately and precisely with reference to underlying muscle activations. It is also easy to retarget motion onto any face model that uses the same semantic structure (Curio et al, 2006). Yet, these FACS derived animations typically present only nonrigid motion -- that is, facial expressions without changes in viewpoint.…”

Evaluation of a Facial Animation Authoring Pipeline Seamlessly Supporting Performance Capture and Manual Key-pose Editing

“…A quantitative evaluation is based on the computation of a numerical difference between the source and the target expressions. Previous results, whether qualitative [23] or quantitative [26], tell us that a perfect match between the source and the target is never achieved. The animation pipeline presented in this work is evaluated, qualitatively, by measuring the recognizability of the six basic facial emotions defined by Ekman [1].…”

“…In performance capture, the puppeteer's face usually does not match the puppet's face: they often have different topology and morphology. The mapping of the puppeteer's facial motion on the puppet face is called retargeting [7,[21][22][23].…”

A new intelligent systems approach to 3D animation in television

“…For instance, some facial animation softwares are based on audio capture [5] and others are based on motion capture [7,6]. Independently of the approach used, it is fundamental that the capture solution assure lip movements synchronization with audio.…”