An Identity-Preserved Model for Face Sketch-Photo Synthesis

Lin, Yandan; Ling, Shenggui; Fu, Keren; Cheng, Peng

doi:10.1109/lsp.2020.3005039

Cited by 26 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They use multiple loss functions to preserve identity and different attributes. Similarly, [25] propose an adversarial model for sketch-photo synthesis, which contains one generator and two discriminators for ensuring image quality and identity preservation. [26] propose a model for both image to sketch and sketch to image conversion that uses Cycle-GAN as its baseline.…”

Section: Image To Image Translation Methodsmentioning

confidence: 99%

Face sketch to photo translation using generative adversarial networks

Farid¹,

Fard²,

Nickabadi³

2021

Preprint

View full text Add to dashboard Cite

Translating face sketches to photo-realistic faces is an interesting and essential task in many applications like law enforcement and the digital entertainment industry. One of the most important challenges of this task is the inherent differences between the sketch and the real image such as the lack of color and details of the skin tissue in the sketch. With the advent of adversarial generative models, an increasing number of methods have been proposed for sketch-to-image synthesis. However, these models still suffer from limitations such as the large number of paired data required for training, the low resolution of the produced images, or the unrealistic appearance of the generated images. In this paper, we propose a method for converting an input facial sketch to a colorful photo without the need for any paired dataset. To do so, we use a pre-trained face photo generating model to synthesize high-quality natural face photos and employ an optimization procedure to keep high-fidelity to the input sketch. We train a network to map the facial features extracted from the input sketch to a vector in the latent space of the face generating model. Also, we study different optimization criteria and compare the results of the proposed model with those of the state-of-the-art models quantitatively and qualitatively. The proposed model achieved 0.655 in the SSIM index and 97.59% rank-1 face recognition rate with higher quality of the produced images.

show abstract

Section: Image To Image Translation Methodsmentioning

confidence: 99%

Face sketch to photo translation using generative adversarial networks

Farid¹,

Fard²,

Nickabadi³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We expect

, which means that after translating various light into a standard one, the synthesized face images

and the corresponding standard illumination

have the same identity i . Identity preservation is very important in various image-to-image translation about face images [ 56 ]. In this paper, H means feature extractor such as ResNet-50, Light-CNN-9 and Light-CNN-29.…”

Section: Proposed Methodsmentioning

confidence: 99%

A High-Performance Face Illumination Processing Method via Multi-Stage Feature Maps

Ling

Lin

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

In recent years, Generative Adversarial Networks (GANs)-based illumination processing of facial images has made favorable achievements. However, some GANs-based illumination-processing methods only pay attention to the image quality and neglect the recognition accuracy, whereas others only crop partial face area and ignore the challenges to synthesize photographic face, background and hair when the original face image is under extreme illumination (Image under extreme illumination (extreme illumination conditions) means that we cannot see the texture and structure information clearly and most pixel values tend to 0 or 255.). Moreover, the recognition accuracy is low when the faces are under extreme illumination conditions. For these reasons, we present an elaborately designed architecture based on convolutional neural network and GANs for processing the illumination of facial image. We use ResBlock at the down-sampling stage in our encoder and adopt skip connections in our generator. This special design together with our loss can enhance the ability to preserve identity and generate high-quality images. Moreover, we use different convolutional layers of a pre-trained feature network to extract varisized feature maps, and then use these feature maps to compute loss, which is named multi-stage feature maps (MSFM) loss. For the sake of fairly evaluating our method against state-of-the-art models, we use four metrics to estimate the performance of illumination-processing algorithms. A variety of experimental data indicate that our method is superior to the previous models under various illumination challenges in illumination processing. We conduct qualitative and quantitative experiments on two datasets, and the experimental data indicate that our scheme obviously surpasses the state-of-the-art algorithms in image quality and identification accuracy.

show abstract

“…It is based on CNN and GAN and realizes inter-domain and intra-domain information transfer to formulate a sketch from the training pairs of photo-viewed sketches. Lin et al [52] and Fang et al [53] presented individual works based on neural networks for face-sketch formulation involving the identity of each subject photo. Yu et al [54] proposed a model to synthesize sketches from photos by GAN that is assisted by composition information of the input photos.…”

Section: Related Workmentioning

confidence: 99%

A Decision Support System for Face Sketch Synthesis Using Deep Learning and Artificial Intelligence

Azhar

Sharif

Raza

et al. 2021

Sensors

View full text Add to dashboard Cite

The recent development in the area of IoT technologies is likely to be implemented extensively in the next decade. There is a great increase in the crime rate, and the handling officers are responsible for dealing with a broad range of cyber and Internet issues during investigation. IoT technologies are helpful in the identification of suspects, and few technologies are available that use IoT and deep learning together for face sketch synthesis. Convolutional neural networks (CNNs) and other constructs of deep learning have become major tools in recent approaches. A new-found architecture of the neural network is anticipated in this work. It is called Spiral-Net, which is a modified version of U-Net fto perform face sketch synthesis (the phase is known as the compiler network C here). Spiral-Net performs in combination with a pre-trained Vgg-19 network called the feature extractor F. It first identifies the top n matches from viewed sketches to a given photo. F is again used to formulate a feature map based on the cosine distance of a candidate sketch formed by C from the top n matches. A customized CNN configuration (called the discriminator D) then computes loss functions based on differences between the candidate sketch and the feature. Values of these loss functions alternately update C and F. The ensemble of these nets is trained and tested on selected datasets, including CUFS, CUFSF, and a part of the IIT photo–sketch dataset. Results of this modified U-Net are acquired by the legacy NLDA (1998) scheme of face recognition and its newer version, OpenBR (2013), which demonstrate an improvement of 5% compared with the current state of the art in its relevant domain.

show abstract

An Identity-Preserved Model for Face Sketch-Photo Synthesis

Cited by 26 publications

References 37 publications

Face sketch to photo translation using generative adversarial networks

Face sketch to photo translation using generative adversarial networks

A High-Performance Face Illumination Processing Method via Multi-Stage Feature Maps

A Decision Support System for Face Sketch Synthesis Using Deep Learning and Artificial Intelligence

Contact Info

Product

Resources

About