Analyzing Clothing Layer Deformation Statistics of 3D Human Motions

Abstract

Recent capture technologies and methods allow not only to retrieve 3D model sequence of moving people in clothing, but also to separate and extract the underlying body geometry, motion component and the clothing as a geometric layer. So far this clothing layer has only been used as raw offsets for individual applications such as retargeting a different body capture sequence with the clothing layer of another sequence, with limited scope, e.g. using identical or similar motions. The structured, semantics and motion-correlated nature of the information contained in this layer has yet to be fully understood and exploited. To this purpose we propose a comprehensive analysis of the statistics of this layer with a simple two-component model, based on PCA subspace reduction of the layer information on one hand, and a generic parameter regression model using neural networks on the other hand, designed to regress from any semantic parameter whose variation is observed in a training set, to the layer parameterization space. We show that this model not only allows to reproduce previous retargeting works, but generalizes the data generation capabilities to other semantic parameters such as clothing variation and size, or physical material parameters with synthetically generated training sequence, paving the way for many kinds of capture data-driven creation and augmentation applications.

Learning to Reconstruct People in Clothing From a Single RGB Camera

“…Shape and clothing have been recovered from RGB images [31,15], depth [16], or scan data [56], but require manual intervention or clothing is limited to a pre-defined set of templates. In [78] a fuzzy vertex association from clothing to body surface is introduced, which allows complex clothing modeled as body offsets. Some works are in-between free-form and model-based methods.…”

Learning to Dress 3D People in Generative Clothing

“…For example, Neophytou and Hilton [35] learn a layered garment model on top of SCAPE [6] from dynamic sequences, but generalization to novel poses is not demonstrated. Yang et al [52] train a neural network to regress a PCA-based representation of clothing, but show generalization on the same sequence or on the same subject. Lähner et al [28] learn a garment-specific pose-deformation model by regressing low-frequency PCA components and high frequency normal maps.…”

“…The mapping from a given body shape and pose to clothing shape is one-to-many. However, existing regression-based clothing models [16,52] produce deterministic results that fail to capture the stochastic nature of clothing deformations. In contrast, we formulate clothing modeling as a probabilistic generation task: for a single pose and body shape, multiple clothing deformations can be sampled.…”

Multi-Garment Net: Learning to Dress 3D People From Images

“…Given a dynamic scan sequence, Neophytou et al [42] learn a two layer model (body and clothing) and use it to dress novel shapes. A similar model has been recently proposed [61], where the clothing layer is associated to the body in a fuzzy fashion. Other methods [60,64] focus explicitly on estimating the body shape under clothing.…”

“…By contrast, we register a single template set to multiple scan instancesvarying in garment geometry, subject identity and pose, which requires a new solution. Most importantly, unlike all previous work [35,47,61], we learn per-garment models and train a CNN to predict body shape and garment geometry directly from images.…”