Realtime Performance-Based Facial Avatars for Immersive Gameplay

Pauly, Mark

doi:10.1145/2522628.2541252

Cited by 7 publications

(4 citation statements)

References 18 publications

(16 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Weise et al, 2011) discusses real time performance based facial animation that enables the user to control the facial expression of a digital avatar in real-time. Similar facial performance based models were proposed by (Pauly, 2013) and (Bermano et al, 2013). On the other hand there is equal effort by scientists to produce facial animation from video or motion capture data.…”

Section: Related Workmentioning

confidence: 86%

Low Bandwidth Video Streaming using FACS, Facial Expression and Animation Techniques

Kumar¹,

Vanualailai²

2016

Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

View full text Add to dashboard Cite

In this paper we describe an easy to use real-time 3D facial expression and animation system that takes the creation of individual facial expressions to the atomic level. That is instead of generating and recording known facial expressions we propose a mechanism that will allow us to create and store each atomic facial distortion. We can then combine some of these singular distortions to create meaningful expressions. FACS Action Units (AUs) is one such technique that describes the simplest visible movement, which cannot be decomposed into more basic ones. We use this as the basis for creating these atomic facial distortions. The Waters muscle based facial model has been used and extended to allow the user to calibrate and record each facial deformation as described in FACS AUs. The user can then create any facial expression by simply stating the series of AUs and its degree of activation in a controlled fashion. These features all form part of the Facial Animation System (FAS). Our FAS is implemented in such a way that enables it to be used as a low bandwidth video streaming player-a real time facial animation player driven only by FACS AUs transmitted as plain text over TCP sockets.

show abstract

Section: Related Workmentioning

confidence: 86%

Low Bandwidth Video Streaming using FACS, Facial Expression and Animation Techniques

Kumar¹,

Vanualailai²

2016

Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

View full text Add to dashboard Cite

show abstract

“…For example, Microsoft has paved the way to controller-free user interaction by releasing Kinect [1]. Other vendors have since then adopted these interaction modalities in their own products, such as Animoji in Apple's iOS, which has been used as Realtime Performance-Based Facial Avatars [9] or BMW's HoloActive touch [4], which enables users to interact with their car's system in hands-free scenarios. Digital technologies are moving towards having the human body at the centre of focus and all the computations should happen either on or around the body.…”

Section: Related Workmentioning

confidence: 99%

Initial Steps Towards Infrastructuring Body-Centric Computing

Pashaei

Smorgun

Lamas

et al. 2019

Human-Computer Interaction – INTERACT 2019

View full text Add to dashboard Cite

Digital technologies are moving towards the human body. Designing for the body is commonly dubbed as body-centric computing (BCC) and includes two sides: 1) the neuro-science side; 2) the technical side mainly focuses on establishing and enabling the interactions. A conceptual framework enabling the systematic design and development of body-centric applications is introduced. Its novelty and importance are assessed within a set of application scenarios.

show abstract

“…A summary of the key evaluation results and method features of the proposed algorithm compared to two previous works is shown in Table 1. Three criteria were considered to compare the systems according to Pauly [40]:Accuracy: The 3D head tracker should estimate the head pose with a high precision compared to a ground truth. Note that this ground truth was generated by applying a third-party commercial software on a public dataset, on which we performed our experiments.…”

Section: Experimental Evaluationmentioning

confidence: 99%

Highly Accurate and Fully Automatic 3D Head Pose Estimation and Eye Gaze Estimation Using RGB-D Sensors and 3D Morphable Models

Ghiass

Arandjelovć

Laurendeau

2018

Sensors

View full text Add to dashboard Cite

This work addresses the problem of automatic head pose estimation and its application in 3D gaze estimation using low quality RGB-D sensors without any subject cooperation or manual intervention. The previous works on 3D head pose estimation using RGB-D sensors require either an offline step for supervised learning or 3D head model construction, which may require manual intervention or subject cooperation for complete head model reconstruction. In this paper, we propose a 3D pose estimator based on low quality depth data, which is not limited by any of the aforementioned steps. Instead, the proposed technique relies on modeling the subject’s face in 3D rather than the complete head, which, in turn, relaxes all of the constraints in the previous works. The proposed method is robust, highly accurate and fully automatic. Moreover, it does not need any offline step. Unlike some of the previous works, the method only uses depth data for pose estimation. The experimental results on the Biwi head pose database confirm the efficiency of our algorithm in handling large pose variations and partial occlusion. We also evaluated the performance of our algorithm on IDIAP database for 3D head pose and eye gaze estimation.

show abstract

Realtime Performance-Based Facial Avatars for Immersive Gameplay

Cited by 7 publications

References 18 publications

Low Bandwidth Video Streaming using FACS, Facial Expression and Animation Techniques

Low Bandwidth Video Streaming using FACS, Facial Expression and Animation Techniques

Initial Steps Towards Infrastructuring Body-Centric Computing

Highly Accurate and Fully Automatic 3D Head Pose Estimation and Eye Gaze Estimation Using RGB-D Sensors and 3D Morphable Models

Contact Info

Product

Resources

About