Lip-synced character speech animation with dominated animeme models

Abstract

One of the holy grails of computer graphics is the generation of photorealistic images with motion data. To re-generate convincing human animations might not be the most challenging part, but it is definitely one of ultimate goals for computer graphics. Amongst full-body human animations, facial animation is the challenging part because of its subtlety and familarity to human beings.In this paper, we like to share the work of lip-sync animation, part of facial animations, as a framework for synthesizing lip-sync character speech animation in real time from a given speech sequence and its corresponding texts.

Synthesizing Expressive Facial and Speech Animation by Text-to-IPA Translation with Emotion Control

“…In particular, phonemes (diphones) and face poses are associated with animation curves predefined by artists to enable easy control over the animation. Lip-sync animation can be applied to a wide range of applications, such as computer games [11] and animation production [12]- [14]. Also, there is a related work focused on incorporating emotion in speech-driven animation [15].…”