DeepWrinkles: Accurate and Realistic Clothing Modeling

Abstract

We present a novel method to generate accurate and realistic clothing deformation from real data capture. Previous methods for realistic cloth modeling mainly rely on intensive computation of physicsbased simulation (with numerous heuristic parameters), while models reconstructed from visual observations typically suffer from lack of geometric details. Here, we propose an original framework consisting of two modules that work jointly to represent global shape deformation as well as surface details with high fidelity. Global shape deformations are recovered from a subspace model learned from 3D data of clothed people in motion, while high frequency details are added to normal maps created using a conditional Generative Adversarial Network whose architecture is designed to enforce realism and temporal consistency. This leads to unprecedented high-quality rendering of clothing deformation sequences, where fine wrinkles from (real) high resolution observations can be recovered. In addition, as the model is learned independently from body shape and pose, the framework is suitable for applications that require retargeting (e.g., body animation). Our experiments show original high quality results with a flexible model. We claim an entirely data-driven approach to realistic cloth wrinkle generation is possible.

Learning‐Based Animation of Clothing for Virtual Try‐On

“…This allows editing the actor's shape and pose parameters while keeping the same captured garment or even changing it. However, re-animated motions lack realism since they cannot predict the nonrigid behavior of clothing under unseen poses or shapes, and are usually limited to copying wrinkles across bodies of different shapes [PMPHB17,LCT18].…”

“…We depict the per-vertex mean error on a static pose (top) and a dynamic sequence (bottom), as we change the body shape over time. To provide a quantitative comparison to existing methods, we additionally show the error suffered by our implementation of cloth retargeting [LCT18,PMPHB17]. As discussed in Section 2, such retargeting methods scale the garment in a way analogous to the body to retain the garment's style.…”

“…In this paper, we propose a learning-based method for cloth animation that meets the needs of virtual try-on, as it models the deformation of a given garment as a function of body motion and shape. Other methods that account for changes in body shape do not deform the garment in a realistic way, and either resize the garment while preserving its style [GRH * 12, BSBC12], or retarget cloth wrinkles to bodies of different shapes [PMPHB17,LCT18].…”

DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare

“…1 (top right) and Fig. 8, a cascaded framework for adding physical simulations of clothing is possible [11,25] and more visually acceptable than end-to-end volumetric reconstruction of BodyNet.…”

Learned Multi-View Texture Super-Resolution

“…For facial animation a deep generative network to infer per-frame texture deformation was introduced [41]. In the context of cloth simulation one can compute a global shape deformation and generate high-frequency normal maps with a GAN [30]. Recently, per-image SR was extended to the multi-view case, by injecting the 3D information in the form of normal maps [34], reaching similar performance as 3D model-based methods.…”