Random sampling for fast face sketch synthesis

Abstract

Exemplar-based face sketch synthesis plays an important role in both digital entertainment and law enforcement. It generally consists of two parts: neighbor selection and reconstruction weight representation. The most time-consuming or main computation complexity for exemplar-based face sketch synthesis methods lies in the neighbor selection process. State-of-the-art face sketch synthesis methods perform neighbor selection online in a data-driven manner by K nearest neighbor (K-NN) searching. Actually, the online search increases the time consuming for synthesis. Moreover, since these methods need to traverse the whole training dataset for neighbor selection, the computational complexity increases with the scale of the training database and hence these methods have limited scalability. In this paper, we proposed a simple but effective offline random sampling in place of online K-NN search to improve the synthesis efficiency. Extensive experiments on public face sketch databases demonstrate the superiority of the proposed method in comparison to state-of-the-art methods, in terms of both synthesis quality and time consumption. The proposed method could be extended to other heterogeneous face image transformation problems such as face hallucination. We release the source codes of our proposed methods and the evaluation metrics for future study online: http://www.ihitworld.com/RSLCR.html.

Semi-supervised Learning for Face Sketch Synthesis in the Wild

“…Considering that patch matching based on traditional image features (e.g., PCA and SIFT) is not robust, a recent method [4] used CNN feature to represent the training patches and computed more accurate combination coefficients. To accelerate the synthesis procedure, Song et al [1] formulated face sketch synthesis as a spatial sketch denoising (SSD) problem, and Wang et al [13] presented an offline random sampling strategy for nearest neighbor selection of patches.…”

“…When evaluating on CUFS, the reference photo-sketch pairs only comes from CUFS, and the same applies to CUFSF. To demonstrate the effectiveness of our model, we compare our results both qualitatively and quantitatively with seven other methods, namely MWF [3], SSD [1], RSLCR [13], DGFL [4], FCN [14], Pix2Pix-GAN [28], and Cycle-GAN [7]. We also compare our results quantitatively with the latest GAN based sketch synthesis methods, i.e., PS 2 -MAN [29] and stack-CA-GAN [18].…”

“…Image Quality Assessment For datasets with ground truth sketches (e.g., CUFS and CUFSF), previous work [13,18,4] typically used structural similarity (SSIM) [31] as an image quality assessment metric to measure the similarity between a generated sketch and the ground truth sketch. However, many researchers (e.g., in super resolution [32] and face sketch synthesis [30,29]) pointed out that SSIM is not always consistent with the perceptual quality.…”

Scoot: A Perceptual Metric for Facial Sketches

“…To minimize the ambiguity of human ranking, we follow the voting strategy [54] to conduct this experi-linear warping lightness shift noise structural damage shifting contrast change blur component lost ghosting checkerboard Figure 6: Our distortions. These distortions are generated by various real synthesis algorithms [27,31,45,53,55,59,64,75,92,93]. ment (∼152K judgments) through the following stages:…”

Fixed pattern noise reduction for infrared images based on cascade residual attention CNN

“…Given this fact, most of the recent researches have focused on developing SB-FPNR methods, such as neural networks (NN) [17], temporal high-pass filter (THPF) [18,19] and constant-statistics (CS) method [20][21]. As for SB-FPNR algorithms, the calibration parameters are iteratively updated by utilizing the information extracted from inter-frame motion, therefore, ghosting artifacts and over smooth effects resulted from the sudden deceleration of scene motion often seriously degrade the noise reduction performance, moreover, the relatively slow convergence process occurred in scene switching is unacceptable for most of the practical applications.In recent years, convolution neural network (CNN) [22] models were explored deeply and applied in various image processing tasks [23], such as image super resolution [24,25], image denoising [26], and sketch synthesis [27][28][29]. To the best of our knowledge, CNN based FPNR methods still have not been extensively investigated in the literature.…”