APDrawingGAN: Generating Artistic Portrait Drawings From Face Photos With Hierarchical GANs

Abstract

Drawing by an artist (a) An image pair in the training set (c) Existing methods APDrawingGAN Headshot Portrait CycleGAN Pix2Pix A test photo Output (b) Our method Deep Image Analogy CNNMRF Gatys The results of using the same input test photo of Barack Obama Figure 1: (a) An artist draws a portrait drawing using a sparse set of lines and very few shaded regions to capture the distinctive appearance of a given face photo. (b) Our APDrawingGAN learns this artistic drawing style and automatically transforms a face photo into a high-quality artistic portrait drawing. (c) Using the same input face photo, six state-of-the-art style transfer methods cannot generate desired artistic drawings: Deep Image Analogy [20], CNNMRF [18], Gatys [11] and Headshot Portrait [32] change facial features or fail to capture style, CycleGAN [40] and Pix2Pix [15] produce false details around hair, eyes or corners of the mouth.

Benchmarking non-photorealistic rendering of portraits

“…Although Table 5 shows two other instances of statistically significant differences between levels (increased attractiveness for XDoG [42] and increased femininity for hedcut [47]) the trends are not consistent across all three levels. Table 6 indicates that the perceived attractiveness of images stylised by APDrawingGAN [43] exhibits a consistently increasing divergence from the original photos across levels, and that this is statistically significant. Although the pebble mosaic stylisation [49] generally TABLE 8 Correlation coefficients between triplet rankings and benchmark levels.…”

“…Of course, this distortion is carried out deliberately to match the geometric style of the artist, so it is natural that this will impact the perception of facial characteristics. APDraw-ingGAN [43] is seen to be sensitive to the complexity of the input; its errors are reasonably low for level 1, but double at level 3 for some characteristics. Table 7 shows that ethnicity is poorly recognised on outputs from the puppet style [44], which is due to low lighting levels causing the shading effect to make the faces dark.…”

“…Yi et al [43] proposed APDrawingGAN, a hierarchical system of generative adversarial networks (GANs) that transforms face photographs into high-quality artistic portrait drawings. Since artists usually use different drawing styles for different facial regions, this hierarchical GAN model combines a global network (for fusing local parts) and six local networks (for individual facial regions).…”

“…The techniques with highest correlation to the levels are neural style transfer [40], which has a tendency to create more spurious facial features (e.g. misplaced eyes) as the images become more cluttered; and both the line drawing methods (artistic sketch method [41] and APDrawingGAN [43]) which often produce fragmented or spurious lines when there are variations in lighting. This user study can be further used to analyse both the benchmark and the NPR algorithms.…”

“…Images from the NPRportrait1.0 benchmark stylised by the artistic sketch method: Berger et al[41] Images from the NPRportrait1.0 benchmark stylised by APDraw-ingGAN: Yi et al[43] …”