Expression Synthesis and Transfer in Parameter Spaces

Abstract

In this paper, we introduce a novel framework that allows users to synthesize the expression of a 3D character by providing a intuitive set of parametric controls. Assuming that human face movements are formed by a set of basis actuation, we analyze a set of real expressions to extract this set together with skin deformation due to the actuation of face. To do this, we first decompose the movement of each marker into a set of distinctive movements. Independent component analysis technique is then adopted to find a independent set of actuations. Our simple and efficient skin deformation model are learned to reproduce the realistic movements of facial parts due to the actuations. In this framework, users can animate characters' faces by controlling the amount actuation or by directly manipulating the face geometry. In addition, the proposed method can be applied to expression transfer which reproduces one character's expression in another character's face. Experimental results demonstrate that our method can produce realistic expression efficiently.

Exploring Non‐Linear Relationship of Blendshape Facial Animation

“…In Shin’s work [SL09], cubic polynomials were used to approximate the vertexes movements due to actuation signals. The polynomial coefficients were learnt to match motion capture data in the case of known actuation signals.…”

“…However in our work, key shapes can be constructed according to specific application, it is more neatly and usable in application. On the other hand, in [SL09], blending weights w i are computed from motion capture data through independent components analysis (ICA). In our work, blending weights together with polynomial coefficients are adjusted in the optimization.…”

“…For example, Joshi has proposed an automatic physically‐motivated scheme that segments blendshapes into smaller regions [JTDP03]. Cao and Shin have used independent components analysis (ICA) to extract a set of meaningful parameters that were independent to each other as much as possible [CFP03, SL09].…”

Using genetic algorithms to uncover individual differences in how humans represent facial emotion

“…Since the introduction of performance-driven animation [16], there have been numerous implementations of data-driven methods, including principle component analysis (PCA) [17,18], Gaussian process latent variable models (GPLVMs) [19,20] and autoencoders [21,22] to create realistic, performance (i.e. participant) driven graphics [23,24]. However, these techniques often rely on the secondary judgement of a skilled artist and are limited in their ability to allow users to explore and interpret resulting graphics.…”

Using evolutionary algorithms to uncover individual differences in how humans represent facial emotion

“…eye shape, mouth curvature, etc.) contribute to a particular facial expression (21,22). GAs have already been used in ecology research and forensic settings, where they enable eyewitnesses to evolve the facial identity of a suspect and generate e-fits with greater accuracy compared to traditional sketches (23,24).…”