Robust Face Sketch Style Synthesis

Abstract

Heterogeneous image conversion is a critical issue in many computer vision tasks, among which example-based face sketch style synthesis provides a convenient way to make artistic effects for photos. However, existing face sketch style synthesis methods generate stylistic sketches depending on many photo-sketch pairs. This requirement limits the generalization ability of these methods to produce arbitrarily stylistic sketches. To handle such a drawback, we propose a robust face sketch style synthesis method, which can convert photos to arbitrarily stylistic sketches based on only one corresponding template sketch. In the proposed method, a sparse representation-based greedy search strategy is first applied to estimate an initial sketch. Then, multi-scale features and Euclidean distance are employed to select candidate image patches from the initial estimated sketch and the template sketch. In order to further refine the obtained candidate image patches, a multi-feature-based optimization model is introduced. Finally, by assembling the refined candidate image patches, the completed face sketch is obtained. To further enhance the quality of synthesized sketches, a cascaded regression strategy is adopted. Compared with the state-of-the-art face sketch synthesis methods, experimental results on several commonly used face sketch databases and celebrity photos demonstrate the effectiveness of the proposed method.

High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

“…Additionally, they employ Markov networks to model the relationship between neighboring patches. Zhang et al [45] employed a sparse representation-based greedy search strategy to first estimate an initial sketch. Candidate image patches from the initial estimated sketch and the template sketch are then selected using multi-scale features.…”

Convolutional Sketch Inversion

“…Generating new training sketch data to match more closely to the sketch style of a specific artist of interest (e.g. by using the method proposed by [11]), and training the network with these sketches would overcome this limitation.…”

“…Few studies developed methods of sketch synthesis to handle more variation in one or more variables at a time, such as lighting [9], and lighting and pose [10]. In a recent study, Zhang et al [11] showed that sketch synthesis by transferring the style of a single sketch could be used also in uncontrolled conditions. In [11], first an initial sketch by a sparse representation-based greedy search strategy was estimated, then candidate patches were selected from a template style sketch and the estimated initial sketch.…”

“…In a recent study, Zhang et al [11] showed that sketch synthesis by transferring the style of a single sketch could be used also in uncontrolled conditions. In [11], first an initial sketch by a sparse representation-based greedy search strategy was estimated, then candidate patches were selected from a template style sketch and the estimated initial sketch. Finally, the candidate patches were refined by a multi-feature-based optimization model and the patches were assembled to produce the final synthesized sketch.…”

Data-Driven Synthesis of Cartoon Faces Using Different Styles

“…learn a feature dictionary from photo patches and replace these patches with a sparse parametric representation during the searching process. Zhang et al [32] develop a framework to synthesize face sketches trained on only one template sketch use a multi-feature-based optimization model to select candidate image patches. The above approaches can create sketches that closely match an input photo, but it is difficult for their frameworks to generate cartoons in given artistic styles, especially some smooth and fancy styles.…”