Interactive facial animation with deep neural networks

Abstract

Creating realistic animations of human faces is still a challenging task in computer graphics. While computer graphics (CG) models capture much variability in a small parameter vector, they usually do not meet the necessary visual quality. This is due to the fact, that geometry‐based animation often does not allow fine‐grained deformations and fails in difficult areas (mouth, eyes) to produce realistic renderings. Image‐based animation techniques avoid these problems by using dynamic textures that capture details and small movements that are not explained by geometry. This comes at the cost of high‐memory requirements and limited flexibility in terms of animation because dynamic texture sequences need to be concatenated seamlessly, which is not always possible and prone to visual artefacts. In this study, the authors present a new hybrid animation framework that exploits recent advances in deep learning to provide an interactive animation engine that can be used via a simple and intuitive visualisation for facial expression editing. The authors describe an automatic pipeline to generate training sequences that consist of dynamic textures plus sequences of consistent three‐dimensional face models. Based on this data, they train a variational autoencoder to learn a low‐dimensional latent space of facial expressions that is used for interactive facial animation.

2D Facial Expression and Movement of Motion for Pain Identification With Deep Learning Methods

“…The findings show that CNN outperforms LBP. In the area of animation, W. Paier et al [42] developed the use of a deep neural network (DNN). The method's central concept is to enable animators to easily manipulate the facial expressions of a virtual human character via the use of deep neural networks.…”

Exploring the influence of generator channel number on the quality of anime-style portrait generation based on DCGAN

“…The art form encompasses a wide range of genres from fantasy to science fiction, each with unique visual characteristics. Signature features of anime-style portraits include large, expressive eyes, elaborate hairstyles and rich color interplay [2,3]. These visual elements not only capture emotion and storytelling but are an important part of narrative and character development in anime culture.…”

Intelligent Animation Creation Method Based on Spatial Separation Perception Algorithm

“…As a crystallization of computer graphics and art, computer animation refers to the use of graphics and image processing techniques to generate a series of scenic images with the help of programming or animation creation software [ 1 ]. Among them, the current frame is a partial modification of the previous frame, and it is a high technology that has developed at a high speed along with computer hardware and graphics algorithms.…”

“…e limitations of traditional animation include too ne division of labor, high equipment requirements, and high cost. e application of traditional animation creation methods has produced many excellent animations, including some animal group animations, but it has some unavoidable limitations such as: (1) it requires a lot of animator's labor. In traditional animation techniques, such as key framing, each character's movement and movement details are designed step by step by highly skilled animators.…”