Cloth and Skin Deformation with a Triangle Mesh Based Convolutional Neural Network

Abstract

We introduce a triangle mesh based convolutional neural network. The proposed network structure can be used for problems where input and/or output are defined on a manifold triangle mesh with or without boundary. We demonstrate its applications in cloth upsampling, adding back details to Principal Component Analysis (PCA) compressed cloth, regressing clothing deformation from character poses, and regressing hand skin deformation from bones' joint angles. The data used for training in this work are generated from high resolution extended position based dynamics (XPBD) physics simulations with small time steps and high iteration counts and from an offline FEM simulator, but it can come from other sources. The inference time of our prototype implementation, depending on the mesh resolution and the network size, can provide between 4 to 134 times faster than a GPU based simulator. The inference also only needs to be done for meshes currently visible by the camera.

Analytically Integratable Zero-restlength Springs for Capturing Dynamic Modes unrepresented by Quasistatic Neural Networks

“…[Patel et al 2020] predicts a low-frequency mesh with a fully connected network and uses a mixture model to add wrinkles. [Chentanez et al 2020] upsamples with graph convolutional neural networks. [Wu et al 2021b] recovers high-frequency geometric details with perturbations of texture.…”

“…and ] use graph networks to learn simulations with both fixed and changing topology. [Chentanez et al 2020] proposes a transition-based model with position and linear/angular velocity of the body as network input (in addition to a state-based model). [Meister et al 2020] uses a fully connected network to predict node-wise acceleration for total Lagrangian explicit dynamics.…”

Learning contact corrections for handle-based subspace dynamics

“…Several other works have modeled deformations driven by skeletal motion, a problem that falls in the scope of the PSD methods described above in Section 2.1. Some of the interesting developments include the use of convolutional networks for mesh-based deformations [Chentanez et al 2020], and robust learning of deformation dynamics under scarce training data [Santesteban et al 2020].…”

Mechanics-aware deformation of yarn pattern geometry

“…Researchers use similar ideas to add fine-scale wrinkle details to a coarse cloth or flesh simulation. Procedural wrinkle methods rely on an underlying strain field [Hadap et al 1999;Rohmer et al 2010;Zuenko and Harders 2019] or explicit simulation of detail [Müller and Chentanez 2010], while data-driven wrinkle methods instead train local operators or pose-dependent systems [Kavan et al 2011;Wang et al 2010;Zurdo et al 2012], with recent works building on recurrent or convolutional neural networks [Chentanez et al 2020;Jin et al 2020;Santesteban et al 2019;Vidaurre et al 2020]. Our work assumes that the input cloth mesh animation already contains all cloth-scale features including wrinkles, and then adds detail purely on a yarn scale.…”