High-fidelity facial and speech animation for VR HMDs

Olszewski, Kyle; Lim, Joseph J.; Saito, Shunsuke; Li, Hao

doi:10.1145/2980179.2980252

Cited by 116 publications

(113 citation statements)

References 46 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…The High Fidelity platform is able to track facial expressions in real time and transfer those to the user's avatar. Despite being able to mimic facial expressions related to speech and emotions, the system itself does not attempt to recognize affect [34]. EEG could extend the high fidelity platform with the ability to adapt to users' affect.…”

Section: Applications In Entertainment and Lifestylementioning

confidence: 99%

Towards emotion recognition for virtual environments: an evaluation of eeg features on benchmark dataset

Menezes

Samara

Galway

et al. 2017

Pers Ubiquit Comput

View full text Add to dashboard Cite

One of the challenges in virtual environments is the difficulty users have in interacting with these increasingly complex systems. Ultimately, endowing machines with the ability to perceive users emotions will enable a more intuitive and reliable interaction. Consequently, using the electroencephalogram as a bio-signal sensor, the affective state of a user can be modelled and subsequently utilised in order to achieve a system that can recognise and react to the user's emotions. This paper investigates features extracted from electroencephalogram signals for the purpose of affective state modelling based on Russell's Circumplex Model. Investigations are presented that aim to provide the foundation for future work in modelling user affect to enhance interaction experience in virtual environments. The DEAP dataset was used within this work, along with a Support Vector Machine and Random Forest, which yielded reasonable classification accuracies for Valence and Arousal using feature vectors based on statistical measurements and band power from the α, β, δ, and θ waves and High Order Crossing of the EEG signal.

show abstract

Section: Applications In Entertainment and Lifestylementioning

confidence: 99%

Towards emotion recognition for virtual environments: an evaluation of eeg features on benchmark dataset

Menezes

Samara

Galway

et al. 2017

Pers Ubiquit Comput

View full text Add to dashboard Cite

show abstract

“…Olszewski et al . [OLSL16] used convolutional neural networks to recover blendshape weights corresponding to the mouth expression of virtual reality headset users. Laine et al .…”

Section: Related Workmentioning

confidence: 99%

Realtime Performance‐Driven Physical Simulation for Facial Animation

Barrielle

Stoiber²

2018

Computer Graphics Forum

View full text Add to dashboard Cite

We present the first realtime method for generating facial animations enhanced by physical simulation from realtime performance capture data. Unlike purely data‐based techniques, our method is able to produce physical effects on the fly through the simulation of volumetric skin behaviour, lip contacts and sticky lips. It remains however practical as it does not require any physical/medical data which are complex to acquire and process, and instead relies only on the input of a blendshapes model. We achieve realtime performance on the CPU by introducing an efficient progressive Projective Dynamics solver to efficiently solve the physical integration steps even when confronted to constantly changing constraints. Also key to our realtime performance is a new Taylor approximation and memoization scheme for the computation of the Singular Value Decompositions required for the simulation of volumetric skin. We demonstrate the applicability of our method by animating blendshape characters from a simple webcam feed .

show abstract

“…Li et al [11] studied the use of a helmet display for facial expression perception. In their follow-up [12] , they also explored the use of helmet displays for face animation and speech synthesis.…”

Section: Helmet Displaymentioning

confidence: 99%

Virtual Fashion Show with HTC VIVE

Xing¹,

Chen²,

Sufang³

et al. 2018

dtcse

View full text Add to dashboard Cite

Abstract. Virtual cloth demonstration has great potential in digital cloth design, and thus has been discussed a lot in academic community. However, limited by the display hardware, existing systems are lack of immersion which affects users' experience a lot. Recently, the emergence of head mounted display system such as Oculus Rift and HTC Vive provides a possible manner to solve the problem. This paper proposes to build a virtual fashion show system to simulate a real show based on HTC Vive. The system use software of unity3D and 3DMax to model the stage scene, lighting, music and character models, clothing to reproduce the real stage maximally, which can give users a high degree of immersion. Simultaneously the system allows users to use the HTC Vive handle to interact with the system, including changing the angle of view and selecting the shape of the stage, lighting, clothing and adjusting the style of model shape. A variety of built-in data (including motion capture data, clothes and human model templates) can intelligently recommend module to meet user needs, which could create a customized fashion show.

show abstract

High-fidelity facial and speech animation for VR HMDs

Cited by 116 publications

References 46 publications

Towards emotion recognition for virtual environments: an evaluation of eeg features on benchmark dataset

Towards emotion recognition for virtual environments: an evaluation of eeg features on benchmark dataset

Realtime Performance‐Driven Physical Simulation for Facial Animation

Virtual Fashion Show with HTC VIVE

Contact Info

Product

Resources

About