Face Sketch Synthesis via Sparse Representation-Based Greedy Search

Abstract

Face sketch synthesis has wide applications in digital entertainment and law enforcement. Although there is much research on face sketch synthesis, most existing algorithms cannot handle some nonfacial factors, such as hair style, hairpins, and glasses if these factors are excluded in the training set. In addition, previous methods only work on well controlled conditions and fail on images with different backgrounds and sizes as the training set. To this end, this paper presents a novel method that combines both the similarity between different image patches and prior knowledge to synthesize face sketches. Given training photo-sketch pairs, the proposed method learns a photo patch feature dictionary from the training photo patches and replaces the photo patches with their sparse coefficients during the searching process. For a test photo patch, we first obtain its sparse coefficient via the learnt dictionary and then search its nearest neighbors (candidate patches) in the whole training photo patches with sparse coefficients. After purifying the nearest neighbors with prior knowledge, the final sketch corresponding to the test photo can be obtained by Bayesian inference. The contributions of this paper are as follows: 1) we relax the nearest neighbor search area from local region to the whole image without too much time consuming and 2) our method can produce nonfacial factors that are not contained in the training set and is robust against image backgrounds and can even ignore the alignment and image size aspects of test photos. Our experimental results show that the proposed method outperforms several state-of-the-arts in terms of perceptual and objective metrics.

Content-Adaptive Sketch Portrait Generation by Decompositional Representation Learning

“…The results generated by our method are satisfying, with fairly perfect shape and subtle detail preserved, while those produced by other methods are nearly unrecognizable. Meanwhile, only SRGS [3] and our methods can produce the non-facial factors, such as hairpin. However, SRGS loses much fine-grained textural detail, such as the hair region of samples in Fig.…”

“…(a) Photos (b) Ours (c) MRF [1] (d) SSD [2] (e) SRGS [3] Despite the widespread applications of sketch portrait, it remains a challenging problem to generate vivid and detailpreserved sketch because of the great difference between photo and sketch. To the best of our knowledge, most of existing approaches generate sketch portraits based on the synthesis of training examples.…”

“…As shown in Fig. 1, the synthesis results of non-facial factors of these examplebased methods are not satisfied, such as hairpins and glasses [1], [3]. Because of the great variations in appearance and geometry of these decorations, it is easy to involve artifacts in the synthesis results.…”

Data-Driven Synthesis of Cartoon Faces Using Different Styles

“…Methods based on Markov random fields are proposed to create sketches from photos by selecting most appropriate neighbor patches to hallucinate a target patch [2], [29], [30]. Zhang et al [31] Face detection rfF----'1• and allcnment .....…”

Face representation learning and its applications on social media

“…Since makeup remove is essentially a novel cross-modal synthesis problem, which shares some basic assumptions with image super-resolution [147] or sketch-photo synthesis [148] problem and dictionary learning based methods have been proved to be successfully in dealing with this kind of tasks. Thus, we briefly review some dictionary learning (DL) works.…”