To enable immersive face-to-face communication in virtual worlds, the facial expressions of a user have to be captured while wearing a virtual reality head-mounted display. Because the face is largely occluded by typical wearable displays, we have designed an HMD that combines ultra-thin strain sensors with a head-mounted RGB-D camera for real-time facial performance capture and animation. AbstractThere are currently no solutions for enabling direct face-to-face interaction between virtual reality (VR) users wearing head-mounted displays (HMDs). The main challenge is that the headset obstructs a significant portion of a user's face, preventing effective facial capture with traditional techniques. To advance virtual reality as a nextgeneration communication platform, we develop a novel HMD that enables 3D facial performance-driven animation in real-time. Our wearable system uses ultra-thin flexible electronic materials that are mounted on the foam liner of the headset to measure surface strain signals corresponding to upper face expressions. These strain signals are combined with a head-mounted RGB-D camera to enhance the tracking in the mouth region and to account for inaccurate HMD placement. To map the input signals to a 3D face model, we perform a single-instance offline training session for each person. For reusable and accurate online operation, we propose a short calibration step to readjust the Gaussian mixture distribution of the mapping before each use. The resulting animations are visually on par with cutting-edge depth sensor-driven facial performance capture systems and hence, are suitable for social interactions in virtual worlds.

No abstract

Recent advancements in the portability and affordability of optical motion capture systems have opened the doors to various clinical applications. In this paper, we look into the potential use of motion capture data for the quantitative analysis of motor symptoms in Parkinson's Disease (PD). The standard of care, human observer-based assessments of the motor symptoms, can be very subjective and are often inadequate for tracking mild symptoms. Motion capture systems, on the other hand, can potentially provide more objective and quantitative assessments. In this pilot study, we perform full-body motion capture of Parkinson's patients with deep brain stimulator off-drugs and with stimulators on and off. Our experimental results indicate that the quantitative measure on spatio-temporal statistics learnt from the motion capture data reveal distinctive differences between mild and severe symptoms. We used a Support Vector Machine (SVM) classifier for discriminating mild vs. severe symptoms with an average accuracy of approximately 90%. Finally, we conclude that motion capture technology could potentially be an accurate, reliable and effective tool for statistical data mining on motor symptoms related to PD. This would enable us to devise more effective ways to track the progression of neurodegenerative movement disorders.

Facial animation often falls short in conveying the nuances present in the facial dynamics of humans. In this article, we investigate the subtleties of the spatial and temporal aspects of eye blinks. Conventional methods for eye blink animation generally employ temporally and spatially symmetric sequences; however, naturally occurring blinks in humans show a pronounced asymmetry on both dimensions. We present an analysis of naturally occurring blinks that was performed by tracking data from high-speed video using active appearance models. Based on this analysis, we generate a set of key-frame parameters that closely match naturally occurring blinks. We compare the perceived naturalness of blinks that are animated based on real data to those created using textbook animation curves. The eye blinks are animated on two characters, a photorealistic model and a cartoon model, to determine the influence of character style. We find that the animated blinks generated from the human data model with fully closing eyelids are consistently perceived as more natural than those created using the various types of blink dynamics proposed in animation textbooks.

No abstract