Lip syncing method for realistic expressive 3D face model

Abstract

Lip synchronization of 3D face model is now being used in a multitude of important fields. It brings a more human, social and dramatic reality to computer games, films and interactive multimedia, and is growing in use and importance. High level of realism can be used in demanding applications such as computer games and cinema. Authoring lip syncing with complex and subtle expressions is still difficult and fraught with problems in terms of realism. This research proposed a lip syncing method of realistic expressive 3D face model. Animated lips requires a 3D face model capable of representing the myriad shapes the human face experiences during speech and a method to produce the correct lip shape at the correct time. The paper presented a 3D face model designed to support lip syncing that align with input audio file. It deforms using Raised Cosine Deformation (RCD) function that is grafted onto the input facial geometry. The face model was based on MPEG-4 Facial Animation (FA) Standard. This paper proposed a method to animate the 3D face model over time to create animated lip syncing using a canonical set of visemes for all pairwise combinations of a reduced phoneme set called ProPhone. The proposed research integrated emotions by the consideration of Ekman model and Plutchik's wheel with emotive eye movements by implementing Emotional Eye Movements Markup Language (EEMML) to produce realistic 3D face model.

Concepts, Methods, and Performances of Particle Swarm Optimization, Backpropagation, and Neural Networks

“…Learning in Neural Networks has made it possible for scientists and researchers to create various applications for multiple industries and create ease in everyday life. The methods reviewed in this paper, namely, Artificial Neural Networks, Backpropagation, and Particle Swarm Optimization have a significant role in Neural Networks in understanding real world problems and tasks, such as image processing, speech or character recognition, and intrusion detection [20,21] Contributions of these methods are significant; however, each category still lacks the definite success as these fields are still progressing and improving on enclosing the gap which exists between its theory and practice. The novelty of the classification concept in Artificial Neural Networks, in particular Particle Swarm Optimization and Backpropagation, means that this field of research is openly, actively, and highly researched every year.…”

Real-Time Cloth Simulation on Virtual Human Character Using Enhanced Position Based Dynamic Framework Technique

“…In this work (8) the physical based simulation is approached from the perspective of low-dimensional linear subspaces like the principal component analysis, which represent the basis vector of the current pose of body and state of cloth, termed the temporal adaptive bases. For creating the low-dimensional bases of the pose space, (5) employed insight from a simulation train data to learn the pose space. However, this approach of learning from simulated data limits the actual dynamics of body pose and cloth state in cloth simulation.…”

“…The raised cosine function (RCF) (13) is adopted for determining the list of constraints + and estimation of positions { , … , }as in (5). By using the RCF, the estimate positions can be modified in such a way that all the constraints of a given cloth fabric, both in a static and dynamic position, is satisfied.…”

“…For applications which require off-line simulations, like in movies, efficiency can be compromised over achieving a more realistic cloth simulation. The real-time applications such as virtual reality in computer games or the garment simulation industry, the reverse is the case (5). The latter is of interest in this paper.…”

“…Particularly, the focus is on synthesizing animation to virtual human character movements which can be made possible through PBD (6). The research utilized the open-source library that is made available in (5). The research further analyzes state-of-the-art real-time cloth simulators such as Syflex, APEX cloth, and Havok cloth which provide cloth simulation solutions for games.…”