Learning skeletal articulations with neural blend shapes

Abstract

Animating a newly designed character using motion capture (mocap) data is a long standing problem in computer animation. A key consideration is the skeletal structure that should correspond to the available mocap data, and the shape deformation in the joint regions, which often requires a tailored, pose-specific refinement. In this work, we develop a neural technique for articulating 3D characters using enveloping with a pre-defined skeletal structure which produces high quality pose dependent deformations. Our framework learns to rig and skin characters with the same articulation structure ( e.g. , bipeds or quadrupeds), and builds the desired skeleton hierarchy into the network architecture. Furthermore , we propose neural blend shapes - a set of corrective pose-dependent shapes which improve the deformation quality in the joint regions in order to address the notorious artifacts resulting from standard rigging and skinning. Our system estimates neural blend shapes for input meshes with arbitrary connectivity, as well as weighting coefficients which are conditioned on the input joint rotations. Unlike recent deep learning techniques which supervise the network with ground-truth rigging and skinning parameters, our approach does not assume that the training data has a specific underlying deformation model. Instead, during training, the network observes deformed shapes and learns to infer the corresponding rig, skin and blend shapes using indirect supervision. During inference, we demonstrate that our network generalizes to unseen characters with arbitrary mesh connectivity, including unrigged characters built by 3D artists. Conforming to standard skeletal animation models enables direct plug-and-play in standard animation software, as well as game engines.

Cleanup Sketched Drawings: Deep Learning-Based Model

“…Various recent approaches have taken advantage of the setting field demonstration for sketch processing by Bessmeltsev and Solomon [ 10 ] on line drawing vectorization. In addition to the approaches for sketch-based models by Iarussi et al [ 14 ] and Li et al [ 15 ].…”

3D mesh pose transfer based on skeletal deformation

“…The automatic generation of the model skeleton and the skinning weights are challenging in computer graphics 20 . The pioneering work was presented by Baran and Popović 21 .…”

“…The user can control the sparsity of the nodes of the generated skeleton by adjusting the parameters but cannot directly control the topology of the generated skeleton. Li et al 20 considered the desired skeleton hierarchy in the network architecture and proposed the Neural blend shape (NBS). They used the edge convolution operators of MeshCNN 27 to build the network.…”

SkinningNet: Two-Stream Graph Convolutional Neural Network for Skinning Prediction of Synthetic Characters

“…Data-driven methods typically require multiple poses of a mesh or different meshes as input to learn how to compute the skinning weights. New methods such as [4,22,32] estimate skinning weights from Motion Capture data. They focus on finding skinning weights of humanoids and assume a fixed skeleton topology, making the network unable to generalize for characters with different skeletons.…”