A neurobehavioural framework for autonomous animation of virtual human faces

Abstract

We describe a neurobehavioural modeling and visual computing framework for the integration of realistic interactive computer graphics with neural systems modelling, allowing real-time autonomous facial animation and interactive visualization of the underlying neural network models. The system has been designed to integrate and interconnect a wide range of computational neuroscience models to construct embodied interactive psychobiological models of behaviour. An example application of the framework combines models of the facial motor system, physiologically based emotional systems, and basic neural systems involved in early interactive behaviour and learning and embodies them in a virtual infant rendered with realistic computer graphics. The model reacts in real time to visual and auditory input and its own evolving internal processes as a dynamic system. The live state of the model which generates the resulting facial behaviour can be visualized through graphs and schematics or by exploring the activity mapped to the underlying neuroanatomy. *The development of autonomous interactive virtual agents and social robots is a developing area of research aiming to recreate naturalistic human behaviour in computer graphic or humanoid robotic form. A key idea behind making humanlike digital characters and robots more realistic in appearance and behaviour is that in doing so we open up many more natural channels of communication -we begin to share the same appearance and signaling mechanisms, making for a more fluid interaction between man and machine, whether it is for entertainment or functionality. However creating realistic appearance and behaviour is challenging

Easy Generation of Facial Animation Using Motion Graphs

“…The author pre‐configures an algorithm which then generates the animation from discrete parameters or events, effectively controlling the animation without directly manipulating the model's transformation/deformation. Procedural animation techniques can be loosely divided into three classes: constraint or rule ‐based [PG96, Per97, BHPN01], where rules impose limits and variations on the generated motion; statistical or knowledge ‐based [KGP02, HG07, SBR*14, HTCH15], with the motions learnt from examples and finally behaviour ‐based, where the cognitive/emotional process is emulated [XMLD07]. In the last, the character's behaviour is modelled to decide the animation, while statistical and constraint‐based methods focus on the actual motion.…”

“…Sagar et al . [SBR*14] combined statistical and behavioural models to create a generative model of facial expressions. They simulated, using, e.g.…”

“…Sagar et al . [SBR*14] presented, arguably, the most advanced work on facial animation generation, however this leads to a complex set‐up, specially for behaviours not previously learned by the model. As a result, the novelties of our method are: A motion graphs approach specifically tuned for facial animation that achieves a compact representation of the original motion.…”

Participatory medicine: model based tools for engaging and empowering the individual

“…To illustrate how these concepts come together, we now describe an experimental psychobiological simulation of an infant, 'BabyX', we are currently developing [39][40][41]. We aim to combine models of the facial and upper body motor systems with models of basic neural systems involved in early interactive behaviour and social learning, to create a virtual infant that can be naturally interacted with.…”

Maintaining and Enhancing Human-Surrogate Presence in Augmented Reality

“…al. introduced an autonomous animated VH face with high-fidelity of graphics and a neurobehavioral model for its realistic behaviors [24]. Hoque et.…”