Facial Attributes Guided Deep Sketch-to-Photo Synthesis

Kazemi, Hadi; Iranmanesh, Mehdi; Dabouei, Ali; Soleymani, Sobhan; Nasrabadi, Nasser M.

doi:10.1109/wacvw.2018.00006

Cited by 33 publications

(23 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Convolutional neural networks have been used as classifiers, but they are also efficient tools to extract and represent discriminative features from the raw data at different levels of abstraction. Compared to hand-crafted features, employing CNN as domain feature extractor demonstrated to be more promising when facing different modalities such as face [26], [27], [28], [29], iris [30] and fingerprint [31], [32]. However, the effects of the fusion at different levels of feature resolution and abstraction and joint optimization of the architecture are not investigated for multimodal biometric identification.…”

Section: Introductionmentioning

confidence: 99%

Multi-Level Feature Abstraction from Convolutional Neural Networks for Multimodal Biometric Identification

Soleymani

Dabouei

Kazemi

et al. 2018

2018 24th International Conference on Pattern Recognition (ICPR)

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a deep multimodal fusion network to fuse multiple modalities (face, iris, and fingerprint) for person identification. The proposed deep multimodal fusion algorithm consists of multiple streams of modality-specific Convolutional Neural Networks (CNNs), which are jointly optimized at multiple feature abstraction levels. Multiple features are extracted at several different convolutional layers from each modality-specific CNN for joint feature fusion, optimization, and classification. Features extracted at different convolutional layers of a modality-specific CNN represent the input at several different levels of abstract representations. We demonstrate that an efficient multimodal classification can be accomplished with a significant reduction in the number of network parameters by exploiting these multi-level abstract representations extracted from all the modality-specific CNNs. We demonstrate an increase in multimodal person identification performance by utilizing the proposed multi-level feature abstract representations in our multimodal fusion, rather than using only the features from the last layer of each modality-specific CNNs. We show that our deep multi-modal CNNs with multimodal fusion at several different feature level abstraction can significantly outperform the unimodal representation accuracy. We also demonstrate that the joint optimization of all the modality-specific CNNs excels the score and decision level fusions of independently optimized CNNs.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Level Feature Abstraction from Convolutional Neural Networks for Multimodal Biometric Identification

Soleymani

Dabouei

Kazemi

et al. 2018

2018 24th International Conference on Pattern Recognition (ICPR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…We compare the proposed method with a number of state-of-the-art approaches that can perform sketch-to-photo transformation such as CycleGAN [11] and Pix2Pix [29]. There are some variations of CycleGAN, such as conditional CycleGAN [38], which utilizes face attributes (e.g., skin color or hair color) as the auxiliary information to improve the network training. However, in our experiments, conditional CycleGAN needs extra information when transforming sketches to photos, and does not show advantage in sketchto-photo transformation and the face identification tasks.…”

Section: A Dataset and Baselinesmentioning

confidence: 99%

Improving Face Sketch Recognition via Adversarial Sketch-Photo Transformation

Han

Shan

et al. 2019

2019 14th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2019)

View full text Add to dashboard Cite

“…However, it still produces slight blur and/or color degraded artifacts. Kazemi et al [19] employ Cycle-GAN conditioned on facial attributes in order to enforce desired facial attributes over the images synthesized from sketches. While sketch-to-face synthesis is a related problem, our unified framework works well with a variety of styles more complex than sketches.…”

Section: Neural Generative Modelsmentioning

confidence: 99%