HairNet: Single-View Hair Reconstruction Using Convolutional Neural Networks

Abstract

We introduce a deep learning-based method to generate full 3D hair geometry from an unconstrained image. Our method can recover local strand details and has real-time performance. State-of-the-art hair modeling techniques rely on large hairstyle collections for nearest neighbor retrieval and then perform ad-hoc refinement. Our deep learning approach, in contrast, is highly efficient in storage and can run 1000 times faster while generating hair with 30K strands. The convolutional neural network takes the 2D orientation field of a hair image as input and generates strand features that are evenly distributed on the parameterized 2D scalp. We introduce a collision loss to synthesize more plausible hairstyles, and the visibility of each strand is also used as a weight term to improve the reconstruction accuracy. The encoder-decoder architecture of our network naturally provides a compact and continuous representation for hairstyles, which allows us to interpolate naturally between hairstyles. We use a large set of rendered synthetic hair models to train our network. Our method scales to real images because an intermediate 2D orientation field, automatically calculated from the real image, factors out the difference between synthetic and real hairs. We demonstrate the effectiveness and robustness of our method on a wide range of challenging real Internet pictures, and show reconstructed hair sequences from videos.

Hair-GAN: Recovering 3D hair structure from a single image using generative adversarial networks

“…The right three columns: the first row is our hair modeling results and the second row is the ground-truth. (the top two rows), [12] (the middle two rows) and [11] (the bottom two rows). For each comparison, from left to right: input image, our results (rough shape, orientation field, 3D hair models), the result of the previous method .…”

“…Inspired by [19], we regard a hairstyle as a fusion of local style patterns distributed around human body, in contrast to [1] [2] [11], where different hairstyles are treated as different combination of styled hair strands. Similar to those previous researches, we collect an original hair dataset with about 300 3D artificial hair models provided by [1], which have already been aligned to an identical bust model.…”

“…Those learned features are sufficiently qualified to describe a new data and to set up a space mapping from the input to the target output, totally independently from those training datum. The method of [1] and the concurrent works of [11] and [12] introduce deep learning into single-view hair modeling. In Chai et al [1]'s method, they use Fig.…”

“…CNN networks to detect hair regions and direction predictors from images as preprocessing for the subsequent hair modeling. Zhou et al [11] use an auto-encoder to directly learn the correspondence from 2D orientation map to hair stand assembly parameterized to a 2D scalp map of low resolution as in [13]. Saito et al [12] put an effort to achieve end-to-end 3D hair inference by leveraging the latent space of auto-encoders to build a bridge between the 2D image and the 3D hair structure.…”

“…In order to build the data for the network trainings, we use 3D artificially synthetic hair database. Like the deep learning methods of [11] and [12], the large amount of database is put aside after the training of the network is completed.…”