The Digital Emily project

Alexander, Oleg; Rogers, Mike; Lambeth, William; Chiang, Matt Jen-Yuan; Debevec, Paul

doi:10.1145/1667239.1667251

Cited by 104 publications

(9 citation statements)

References 21 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although there have been great advances in modeling, rigging, rendering, motion capture and retargeting techniques -with the goal of escaping the "Uncanny Valley" (Alexander, Rogers, Lambeth, Chiang, & Debevec, 2009;McDonnell, Breidt, & Bülthoff, 2012) -the creation of realistic and convincing face behaviors for games and movies is still strongly dependent on animator skills. There are some studies about how to convey and evaluate a character's complex facial behaviors (Paleari & Lisetti, 2006;Bevacqua, Mancini, Niewiadomski, & Pelachaud, 2007;Rehm, 2008;Orvalho & Sousa, 2009;Niewiadomski, Hyniewska, & Pelachaud, 2009;Queiroz, Braun, et al, 2010;Demeure, Niewiadomski, & Pelachaud, 2011;de Melo, Carnevale, & Gratch, 2011;Xolocotzin Eligio, Ainsworth, & Crook, 2012), but few really focus on microexpressions (Zielke, Dufour, & Hardee, 2011).…”

Section: Introductionmentioning

confidence: 99%

Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Queiroz

Musse

Badler

2014

Presence: Teleoperators and Virtual Environments

View full text Add to dashboard Cite

Due to varied personal, social, or even cultural situations, people sometimes conceal or mask their true emotions. These suppressed emotions can be expressed in a very subtle way by brief movements called microexpressions. We investigate human subjects' perception of hidden emotions in virtual faces, inspired by recent psychological experiments. We created animations with virtual faces showing some facial expressions and inserted brief secondary expressions in some sequences, in order to try to convey a subtle second emotion in the character. Our evaluation methodology consists of two sets of experiments, with three different sets of questions. The first experiment verifies that the accuracy and concordance of the participant's responses with synthetic faces matches the empirical results done with photos of real people in the paper by X.-b. Shen, Q. Wu, and X.-l. Fu, 2012, "Effects of the duration of expressions on the recognition of microexpressions," Journal of Zhejiang University Science B, 13(3), 221-230. The second experiment verifies whether participants could perceive and identify primary and secondary emotions in virtual faces. The third experiment tries to evaluate the participant's perception of realism, deceit, and valence of the emotions. Our results show that most of the participants recognized the foreground (macro) emotion and most of the time they perceived the presence of the second (micro) emotion in the animations, although they did not identify it correctly in some samples. This experiment exposes the benefits of conveying microexpressions in computer graphics characters, as they may visually enhance a character's emotional depth through subliminal microexpression cues, and consequently increase the perceived social complexity and believability (Shen, Wu, & Fu, 2012). The second experiment verifies if participants could perceive and identify primary and secondary emotions in virtual faces. The third tries to evaluate the participant's perception of realism, deceit and valence of the emotions. Our results show that most of the participants recognized the foreground (macro) emotion and most of the time they had perceived the presence of the second (micro) emotion in the animations, although they did not identify it correctly in some samples. This experiment exposes the benefits of conveying microexpressions in Computer Graphics characters, as they may visually enhance a character's emotional "depth" through subliminal microexpression cues and consequently increase its perceived social complexity and believability.

show abstract

Section: Introductionmentioning

confidence: 99%

Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Queiroz

Musse

Badler

2014

Presence: Teleoperators and Virtual Environments

View full text Add to dashboard Cite

show abstract

“…9 (a). The Face2Face [33] transfers facial expressions inferred by 3D face morphable model [64], [65], [66], retrieves mouth texture by the facial expression and renders the talking face. Our method produces competitive results with realistic texture and mouth movement, suggesting that our Style Translation Network learns accurate mouth movement from the input audio.…”

Section: ) Comparisons To Audio/text Driven Dubbing Methodsmentioning

confidence: 99%

Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis

Song

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Existing automated dubbing methods are usually designed for Professionally Generated Content (PGC) production, which requires massive training data and training time to learn a person-specific audio-video mapping. In this paper, we investigate an audio-driven dubbing method that is more feasible for User Generated Content (UGC) production. There are two unique challenges to design a method for UGC: 1) the appearances of speakers are diverse and arbitrary as the method needs to generalize across users; 2) the available video data of one speaker are very limited. In order to tackle the above challenges, we first introduce a new Style Translation Network to integrate the speaking style of the target and the speaking content of the source via a cross-modal AdaIN module. It enables our model to quickly adapt to a new speaker. Then, we further develop a semi-parametric video renderer, which takes full advantage of the limited training data of the unseen speaker via a videolevel retrieve-warp-refine pipeline. Finally, we propose a temporal regularization for the semi-parametric renderer, generating more continuous videos. Extensive experiments show that our method generates videos that accurately preserve various speaking styles, yet with considerably lower amount of training data and training time in comparison to existing methods. Besides, our method achieves a faster testing speed than most recent methods.

show abstract

“…A 3D morphable face model (3DMM) produces vector space representations that capture various facial attributes such as shape, expression and pose [6,4,8,15,16]. Although the previous 3DMM methods [6,4,8] have limitations in estimating face texture and lighting conditions accurately, recent methods [15,16] overcome these limitations. We utilize the state-of-the-art 3DMM [16] to effectively capture the various facial attributes and supervise our model.…”

Section: Related Workmentioning

confidence: 99%

MFIM: Megapixel Facial Identity Manipulation

Na¹

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Face swapping is a task that changes a facial identity of a given image to that of another person. In this work, we propose a novel face-swapping framework called Megapixel Facial Identity Manipulation (MFIM). The face-swapping model should achieve two goals. First, it should be able to generate a high-quality image. We argue that a model which is proficient in generating a megapixel image can achieve this goal. However, generating a megapixel image is generally difficult without careful model design. Therefore, our model exploits pretrained StyleGAN in the manner of GAN-inversion to effectively generate a megapixel image. Second, it should be able to effectively transform the identity of a given image. Specifically, it should be able to actively transform ID attributes (e.g., face shape and eyes) of a given image into those of another person, while preserving ID-irrelevant attributes (e.g., pose and expression). To achieve this goal, we exploit 3DMM that can capture various facial attributes. Specifically, we explicitly supervise our model to generate a face-swapped image with the desirable attributes using 3DMM. We show that our model achieves state-of-the-art performance through extensive experiments. Furthermore, we propose a new operation called ID mixing, which creates a new identity by semantically mixing the identities of several people. It allows the user to customize the new identity.

show abstract

The Digital Emily project

Cited by 104 publications

References 21 publications

Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces

Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis

MFIM: Megapixel Facial Identity Manipulation

Contact Info

Product

Resources

About