Synthesizing cloth wrinkles by CNN‐based geometry image superresolution

Abstract

We propose a novel deep learning-based method, called mesh superresolution, to enrich low-resolution (LR) cloth meshes with wrinkles. A pair of low and high-resolution (HR) meshes are simulated, with the simulation of the HR mesh tracks with that of the LR mesh. The frame data are converted into geometry images and used as a training data set. A residual network, called SR residual network, is employed to train an image synthesizer that superresolves an LR image into an HR one. Once the HR image is converted back to an HR mesh, it is abundant in wrinkles compared with its coarse counterpart. The synthesizing is very efficient and is 24× faster than a full HR simulation. We demonstrate the performances of mesh superresolution with various simulation scenes.

Reconstruction of a fluttering flag from a single image

“…Those methods consider the cloth animation in virtual 3 D space, in which the cloth or flag has a definitive 3 D mesh structure. Chen et al 26 proposed a mesh super-resolution to enrich low reolsution cloth meshes with wrinkes based on deep learning, which uses the SRResNet to train an image synthesizer. However, our work focus on driving the virtual flag to flutter with a reconstructed approximate mesh from a single image.…”

Cloth and Skin Deformation with a Triangle Mesh Based Convolutional Neural Network

“…In computer vision, Lähner et al [LCT18] uses a Pix2Pix [IZZE17] network to learn to upsample the normal map image of a coarse simulation to real‐world captured cloth wrinkles. Chen et al [CYJ ∗ 18] trained a CNN akin to image super resolution to upsample low resolution cloth to high resolution in texture space. Oh et al [OLL18] train fully connected networks (FCNs) for subdividing a triangle into 4 triangles and upsampling the cloth simulation.…”

Enabling garment-agnostic laundry tasks for a Robot Household Companion

“…Figure 5. 5 WiLD descriptor for flat and curved regions Figure 5. 6 Different steps of the path extraction stage Figure 5.…”

“…4 Folding paths supported by the simulator Figure 6. 5 Examples randomly sampled from the synthetic folding dataset . .…”