Superpixel-Based Face Sketch–Photo Synthesis

Peng, Chunlei; Gao, Xinbo; Wang, Nannan; Li, Jie

doi:10.1109/tcsvt.2015.2502861

Cited by 59 publications

(31 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wang et al [34] categorize photo-sketch synthesis methods based on model construction techniques into three main classes: 1) subspace learning-based, 2) sparse representation-based, and 3) Bayesian inference-based approaches. Peng et al [20] perform the categorization based on representation strategies and come up with three broad approaches: 1) holistic image-based, 2) independent local patch-based, and 3) local patch with spatial constraintsbased methods.…”

Section: A Face Photo-sketch Synthesismentioning

confidence: 99%

“…These candidate patches are refined and assembled to obtain the final sketch which is further enhanced using a cascaded regression strategy. Peng et al [20] proposed a superpixelbased synthesis method involving two stage synthesis procedure. Wang et al [31] recently proposed the use of Bayesian framework consisting of neighbor selection model and weight computation model.…”

Section: A Face Photo-sketch Synthesismentioning

confidence: 99%

See 1 more Smart Citation

High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

Wang

Sindagi

Patel

2018

2018 13th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2018)

129

View full text Add to dashboard Cite

Synthesizing face sketches from real photos and its inverse have many applications. However, photo/sketch synthesis remains a challenging problem due to the fact that photo and sketch have different characteristics. In this work, we consider this task as an image-to-image translation problem and explore the recently popular generative models (GANs) to generate high-quality realistic photos from sketches and sketches from photos. Recent GANbased methods have shown promising results on image-toimage translation problems and photo-to-sketch synthesis in particular, however, they are known to have limited abilities in generating high-resolution realistic images. To this end, we propose a novel synthesis framework called Photo-Sketch Synthesis using Multi-Adversarial Networks, (PS 2 -MAN) that iteratively generates low resolution to high resolution images in an adversarial way. The hidden layers of the generator are supervised to first generate lower resolution images followed by implicit refinement in the network to generate higher resolution images. Furthermore, since photosketch synthesis is a coupled/paired translation problem, we leverage the pair information using CycleGAN framework. Both Image Quality Assessment (IQA) and Photo-Sketch Matching experiments are conducted to demonstrate the superior performance of our framework in comparison to existing state-of-the-art solutions. Code available at: https://github.com/lidan1/PhotoSketchMAN.

show abstract

Section: A Face Photo-sketch Synthesismentioning

confidence: 99%

Section: A Face Photo-sketch Synthesismentioning

confidence: 99%

High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

Wang

Sindagi

Patel

2018

2018 13th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2018)

129

View full text Add to dashboard Cite

show abstract

“…Sketch-photo Face Synthesis Sketch-photo face synthesis is now quite well studied [12,18]. Existing studies can be categorised according to whether they use classic [14,15,19] or deep [5,6] methods; and whether they process images holistically [5,6] or patch-wise [14,15,19] (more common for deep and classic methods respectively).…”

Section: Related Workmentioning

confidence: 99%

“…A particularly interesting variant is that of synthesising photos based on facial sketches, which has applications in entertainment and law enforcement [18]. In the past decade, this problem has been well studied, and promising results have been achieved using both patch-based [14,15,19] and, more recently, deep learning-based [5] approaches.…”

Section: Introductionmentioning

confidence: 99%

Now You See Me: Deep Face Hallucination for Unviewed Sketches

Hu¹,

Li²,

Song³

et al. 2017

Procedings of the British Machine Vision Conference 2017

View full text Add to dashboard Cite

Face hallucination has been well studied in the last decade because of its useful applications in law enforcement and entertainment. Promising results on the problem of sketch-photo face hallucination have been achieved with classic, and increasingly deep learning-based methods. However, synthesized photos still lack the crisp fidelity of real photos. More importantly, good results have primarily been demonstrated on very constrained datasets where the style variability is very low, and crucially the sketches are perfectly align-able traces of the ground-truth photos. However, realistic applications in entertainment or law enforcement require working with more unconstrained sketches drawn from memory or description, which are not rigidly align-able. In this paper, we develop a new deep learning approach to address these settings. Our image-image regression network is trained with a combination of content and adversarial losses to generate crisp photorealistic images, and it contains an integrated spatial transformer network to deal with non-rigid alignment between the domains. We evaluate face synthesis on classic constrained, as well as unviewed, benchmarks namely CUHK, MGDB, and FSMD. The results qualitatively and quantitatively outperform existing approaches. 2HU, LI, SONG, HOSPEDALES: DEEP FACE HALLUCINATION FOR UNVIEWED SKETCHESThe standard viewed-sketch databases are also very constrained, in that there is little variability in conditions such as background, sketch style, and even subject ethnicity (CUHK). However, neither of these assumptions hold in real law or entertainment applications of sketch-photo synthesis. Here, the sketches and photos are more unconstrained, and crucially artists are drawing from their imagination, or description. This means that the sketches are affected by communication and memory imperfections [3,13] as well as the conventional sketch-photo modality gap. So photo hallucination is now a much more complicated mapping than simple colour texturing after rigid alignment. This can be seen in the results of the few studies that test on unviewed forensic sketches after training on viewed benchmarks: The quality of the synthesis results in the unviewed case is much worse [5,14].In this paper we develop a powerful deep learning-based method for sketch-photo face hallucination that produces more crisp images than prior work while addressing the less constrained unviewed setting, that is harder but more practically relevant. We build upon a fully convolutional image-image regression network [5] that can provide a rich non-linear mapping from sketches to photos. To make this mapping learnable, given the lack of a rigid alignment between photos and sketches in the unviewed case, we integrate a modified spatial transformer network (STN) [10] into the regressor. Our STN network inputs facial geometry defined by detected facial interest points, and non-rigidly warps the sketch and photo into alignment. To enable the synthesis of high fidelity crisp photos, we first extend the imageimage regres...

show abstract

“…Digital image processing (DIP) is widely used in various research areas, such as medical image processing [1], biology [2], physics [3,4], and astronomy [5], as well as in the industrial [6], defense, and law enforcement fields [7]. Image denoising and compression are valuable tasks of the DIP [8], and various approaches are used to solve these problems, the most common of which are the Fourier transform [9] and the wavelet transform [10][11][12], and a special hardware is widely used.…”

Section: Introductionmentioning

confidence: 99%

Analysis of the Quantization Noise in Discrete Wavelet Transform Filters for Image Processing

et al. 2018

View full text Add to dashboard Cite

Abstract:In this paper, we analyze the noise quantization effects in coefficients of discrete wavelet transform (DWT) filter banks for image processing. We propose the implementation of the DWT method, making it possible to determine the effective bit-width of the filter banks coefficients at which the quantization noise does not significantly affect the image processing results according to the peak signal-to-noise ratio (PSNR). The dependence between the PSNR of the DWT image quality on the wavelet and the bit-width of the wavelet filter coefficients is analyzed. The formulas for determining the minimal bit-width of the filter coefficients at which the processed image achieves high quality (PSNR ≥ 40 dB) are given. The obtained theoretical results were confirmed through the simulation of DWT for a test image using the calculated bit-width values. All considered algorithms operate with fixed-point numbers, which simplifies their hardware implementation on modern devices: field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), etc.

show abstract

Superpixel-Based Face Sketch–Photo Synthesis

Cited by 59 publications

References 29 publications

High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

Now You See Me: Deep Face Hallucination for Unviewed Sketches

Analysis of the Quantization Noise in Discrete Wavelet Transform Filters for Image Processing

Contact Info

Product

Resources

About