An MPEG-4 facial animation system driven by synthetic speech

Abstract

This paper' provides an overview of the activities that have led to the development of a prototype application of a "talking head" compliant with MPEG-4 spectfication. The work done so far Jits in the context of one of the recent work-items defined by the ISOBEC JTCI/SC29/WGll worldwide known as MPEG. This IS0 Working Group is now in the process of defining a unifying framework where natural and synthetic audiovisual objects can be combined and rendered as a unique bouquet of synchronised interactive media. The developed facial animation system implements the following features: animation of predefined / downloaded face models, animation of face models driven either by speech synthesis applications or by MPEG-4 animation parameters. To improve photo-realism, face models can be texture mapped and calibrated according to the countenances of real people.

A Web3D toolbox for creating H-Anim compatible actors

“…A special Animation node has been implemented for the H-ANIM TOOLBOX (using the Shout3D API) which encapsulates the scripting of (several) behaviour sequences 6 and handles the dynamic generation (and deletion) of the required routes, as described in [7]. For seamless models a special node (AnimationSeamless) including an integrated deformation engine can be used (with the same animation data).…”

“…the face geometry and influence the deformation of certain face regions. 8 The Face Animation Parameters (FAP) defined in the MPEG-4 standard are especially well suited as they can be used to create arbitrary facial expressions [6,11,12].…”

“…If used for VRML geometries it has the same disadvantages as with metaballs 4. This calibration technique is based on[6] and requires manual identification of a subset of the MPEG-4 facial feature points in the picture. depends on the predefined face template which can be selected.…”

“…A motion capture system for VRML humanoids is described in[8] 6. For each behaviour a special Behaviour node (e.g.…”