High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

Wang, Lidan; Sindagi, Vishwanath A.; Patel, Vishal M.

doi:10.1109/fg.2018.00022

Cited by 129 publications

(104 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though we have only evaluated our approach on polarimetric thermal to visible face verification, in the future, we will evaluate the performance of this method on other heterogeneous face recognition tasks such as sketch to face matching [32].…”

Section: Resultsmentioning

confidence: 99%

Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis

Riggan

et al. 2019

2019 International Conference on Biometrics (ICB)

View full text Add to dashboard Cite

Polarimetric thermal to visible face verification entails matching two images that contain significant domain differences. Several recent approaches have attempted to synthesize visible faces from thermal images for cross-modal matching. In this paper, we take a different approach in which rather than focusing only on synthesizing visible faces from thermal faces, we also propose to synthesize thermal faces from visible faces. Our intuition is based on the fact that thermal images also contain some discriminative information about the person for verification. Deep features from a pre-trained Convolutional Neural Network (CNN) are extracted from the original as well as the synthesized images. These features are then fused to generate a template which is then used for verification. The proposed synthesis network is based on the self-attention generative adversarial network (SAGAN) which essentially allows efficient attention-guided image synthesis. Extensive experiments on the ARL polarimetric thermal face dataset demonstrate that the proposed method achieves state-of-the-art performance.

show abstract

Section: Resultsmentioning

confidence: 99%

Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis

Riggan

et al. 2019

2019 International Conference on Biometrics (ICB)

View full text Add to dashboard Cite

show abstract

“…To demonstrate the effectiveness of our model, we compare our results both qualitatively and quantitatively with seven other methods, namely MWF [3], SSD [1], RSLCR [13], DGFL [4], FCN [14], Pix2Pix-GAN [28], and Cycle-GAN [7]. We also compare our results quantitatively with the latest GAN based sketch synthesis methods, i.e., PS 2 -MAN [29] and stack-CA-GAN [18]. Since the models of their work are not available, we can only compare with the results that are directly taken from their published papers.…”

Section: Evaluation On Public Benchmarksmentioning

confidence: 95%

“…Image Quality Assessment For datasets with ground truth sketches (e.g., CUFS and CUFSF), previous work [13,18,4] typically used structural similarity (SSIM) [31] as an image quality assessment metric to measure the similarity between a generated sketch and the ground truth sketch. However, many researchers (e.g., in super resolution [32] and face sketch synthesis [30,29]) pointed out that SSIM is not always consistent with the perceptual quality. One main reason is that SSIM favors slightly blurry images when the images contain rich textures.…”

Section: Quantitative Comparisonmentioning

confidence: 99%

“…Comparison with PS 2 -MAN and stack-CA-GAN To further demonstrate the effectiveness of the proposed method, we compare it with two latest GAN methods, namely PS 2 -MAN [29] and stack-CA-GAN [18], which are specially designed for sketch synthesis. As shown in Table 2, our method achieves the best performance on almost all datasets, except for the SSIM score in CUFSF.…”

Section: Quantitative Comparisonmentioning

confidence: 99%

See 1 more Smart Citation

Semi-supervised Learning for Face Sketch Synthesis in the Wild

Chen

Liu

Tan

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Face sketch synthesis has made great progress in the past few years. Recent methods based on deep neural networks are able to generate high quality sketches from face photos. However, due to the lack of training data (photo-sketch pairs), none of such deep learning based methods can be applied successfully to face photos in the wild. In this paper, we propose a semi-supervised deep learning architecture which extends face sketch synthesis to handle face photos in the wild by exploiting additional face photos in training. Instead of supervising the network with ground truth sketches, we first perform patch matching in feature space between the input photo and photos in a small reference set of photo-sketch pairs. We then compose a pseudo sketch feature representation using the corresponding sketch feature patches to supervise our network. With the proposed approach, we can train our networks using a small reference set of photo-sketch pairs together with a large face photo dataset without ground truth sketches. Experiments show that our method achieves state-of-the-art performance both on public benchmarks and face photos in the wild. Codes are available at https://github.com/chaofengc/Face-Sketch-Wild.

show abstract

“…Several works such as [14,15,16,17] have demonstrated that different sized objects are captured by different layers in a deep network. Hence, an obvious approach would be to design a multi-scale counting network [18,19] that concatenates feature maps from different layers of the VGG16 network. However, earlier layers in a deep network capture primitive features and do not learn semantic awareness.…”

Section: Introductionmentioning

confidence: 99%

HA-CCN: Hierarchical Attention-Based Crowd Counting Network

Sindagi

Patel

2020

IEEE Trans. on Image Process.

179

View full text Add to dashboard Cite

Single image-based crowd counting has recently witnessed increased focus, but many leading methods are far from optimal, especially in highly congested scenes. In this paper, we present Hierarchical Attention-based Crowd Counting Network (HA-CCN) that employs attention mechanisms at various levels to selectively enhance the features of the network. The proposed method, which is based on the VGG16 network, consists of a spatial attention module (SAM) and a set of global attention modules (GAM). SAM enhances low-level features in the network by infusing spatial segmentation information, whereas the GAM focuses on enhancing channel-wise information in the higher level layers. The proposed method is a single-step training framework, simple to implement and achieves state-of-the-art results on different datasets. Furthermore, we extend the proposed counting network by introducing a novel set-up to adapt the network to different scenes and datasets via weak supervision using image-level labels. This new set up reduces the burden of acquiring labour intensive point-wise annotations for new datasets while improving the cross-dataset performance.

show abstract

High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

Cited by 129 publications

References 49 publications

Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis

Polarimetric Thermal to Visible Face Verification via Self-Attention Guided Synthesis

Semi-supervised Learning for Face Sketch Synthesis in the Wild

HA-CCN: Hierarchical Attention-Based Crowd Counting Network

Contact Info

Product

Resources

About