Real-time data driven deformation using kernel canonical correlation analysis

Feng, Wei-Wen; Kim, Byung-Uck; Yu, Yizhou

doi:10.1145/1360612.1360690

Cited by 58 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, an animator would need to learn to use the fitted bone structure in order to author new animations. Example-based skinning methods have been developed [Feng et al 2008;Mohr and Gleicher 2003;Mukai and Kuriyama 2016;Wang et al 2007] that uses the original skeleton from a character rig. These methods use training examples to learn a deformation model that can approximate mesh deformations given new skeleton poses not seen during model training.…”

Section: Related Workmentioning

confidence: 99%

Fast and deep deformation approximations

et al. 2018

View full text Add to dashboard Cite

Character rigs are procedural systems that compute the shape of an animated character for a given pose. They can be highly complex and must account for bulges, wrinkles, and other aspects of a character's appearance. When comparing film-quality character rigs with those designed for real-time applications, there is typically a substantial and readily apparent difference in the quality of the mesh deformations. Real-time rigs are limited by a computational budget and often trade realism for performance. Rigs for film do not have this same limitation, and character riggers can make the rig as complicated as necessary to achieve realistic deformations. However, increasing the rig complexity slows rig evaluation, and the animators working with it can become less efficient and may experience frustration. In this paper, we present a method to reduce the time required to compute mesh deformations for film-quality rigs, allowing better interactivity during animation authoring and use in real-time games and applications. Our approach learns the deformations from an existing rig by splitting the mesh deformation into linear and nonlinear portions. The linear deformations are computed directly from the transformations of the rig's underlying skeleton. We use deep learning methods to approximate the remaining nonlinear portion. In the examples we show from production rigs used to animate lead characters, our approach reduces the computational time spent on evaluating deformations by a factor of 5X-10X. This significant savings allows us to run the complex, film-quality rigs in real-time even when using a CPU-only implementation on a mobile device.

show abstract

Section: Related Workmentioning

confidence: 99%

Fast and deep deformation approximations

et al. 2018

View full text Add to dashboard Cite

show abstract

“…In both cases, the mesh and its associated skeleton at rest are not sufficient, and further human intervention is required. An interesting discussion on examplebased deformation methods, can be found in [17]. In the next section, we discuss several volume preservation methods for skinned characters, which have been proposed to tackle the loss of volume artefact of linear skinning techniques.…”

Section: Example-based Skinning Methodsmentioning

confidence: 99%

Skin Deformation Methods for Interactive Character Animation

Rumman

Fratarcangeli

2017

Communications in Computer and Information Science

View full text Add to dashboard Cite

Character animation is a vital component of contemporary computer games, animated feature films and virtual reality applications. The problem of creating appealing character animation can best be described by the title of the animation bible: "The Illusion of Life". The focus is not on completing a given motion task, but more importantly on how this motion task is performed by the character. This does not necessarily require realistic behavior, but behavior that is believable. This of course includes the skin deformations when the character is moving. In this paper, we focus on the existing research in the area of skin deformation, ranging from skeleton-based deformation and volume preserving techniques to physically based skinning methods. We also summarize the recent contributions in deformable and soft body simulations for articulated characters, and discuss various geometric and example-based approaches.

show abstract

“…Similarly, Le and Deng [2014] also finds a SSD representation of the deformation, but organized in a skeletal hierarchy for easier animation afterward. Feng et al [2008] learns a skinned mesh via SSD from example data in order to animate with control points. Because a bone-based deformation system is not the best way to represent facial deformations, these methods alone are not suitable for our purposes.…”

Section: Related Workmentioning

confidence: 99%

Fast and deep facial deformations

et al. 2020

View full text Add to dashboard Cite

Film-quality characters typically display highly complex and expressive facial deformation. The underlying rigs used to animate the deformations of a character's face are often computationally expensive, requiring high-end hardware to deform the mesh at interactive rates. In this paper, we present a method using convolutional neural networks for approximating the mesh deformations of characters' faces. For the models we tested, our approximation runs up to 17 times faster than the original facial rig while still maintaining a high level of fidelity to the original rig. We also propose an extension to the approximation for handling high-frequency deformations such as fine skin wrinkles. While the implementation of the original animation rig depends on an extensive set of proprietary libraries making it difficult to install outside of an in-house development environment, our fast approximation relies on the widely available and easily deployed TensorFlow libraries. In addition to allowing high frame rate evaluation on modest hardware and in a wide range of computing environments, the large speed increase also enables interactive inverse kinematics on the animation rig. We demonstrate our approach and its applicability through interactive character posing and real-time facial performance capture.

show abstract

Real-time data driven deformation using kernel canonical correlation analysis

Cited by 58 publications

References 33 publications

Fast and deep deformation approximations

Fast and deep deformation approximations

Skin Deformation Methods for Interactive Character Animation

Fast and deep facial deformations

Contact Info

Product

Resources

About