Fast Face Image Synthesis With Minimal Training

Banerjee, S.; Scheirer, Walter J.; Bowyer, Kevin W.; Flynn, Patrick J.

doi:10.1109/wacv.2019.00230

Cited by 11 publications

(22 citation statements)

References 64 publications

(123 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specific to head pose, lighting and expression agnostic face recognition, approaches like feature normalization using class centers [69,76] and class separation using angular margins [36,68,12] have been proposed. Such recognition tasks have also benefitted from mixing samples from different domains, like real and synthetic [40,2].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Driver Glance Classification In-the-wild: Towards Generalization Across Domains and Subjects

Banerjee¹,

Joshi²,

Turcot³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Distracted drivers are dangerous drivers. Equipping advanced driver assistance systems (ADAS) with the ability to detect driver distraction can help prevent accidents and improve driver safety. In order to detect driver distraction, an ADAS must be able to monitor their visual attention. We propose a model that takes as input a patch of the driver's face along with a crop of the eye-region and classifies their glance into 6 coarse regions-of-interest (ROIs) in the vehicle. We demonstrate that an hourglass network, trained with an additional reconstruction loss, allows the model to learn stronger contextual feature representations than a traditional encoder-only classification module. To make the system robust to subject-specific variations in appearance and behavior, we design a personalized hourglass model tuned with an auxiliary input representing the driver's baseline glance behavior. Finally, we present a weakly supervised multi-domain training regimen that enables the hourglass to jointly learn representations from different domains (varying in camera type, angle), utilizing unlabeled samples and thereby reducing annotation cost.

show abstract

Section: Related Workmentioning

confidence: 99%

“…where N is the training set in a batch and C the ground truth 2 Model architecture details can be found in Section 7…”

Section: Two-channel Hourglassmentioning

confidence: 99%

Driver Glance Classification In-the-wild: Towards Generalization Across Domains and Subjects

Banerjee¹,

Joshi²,

Turcot³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To augment the sparsely populated classes, researchers have synthesized artificial data that mimic features of the real samples from these classes. For example, the same facial texture can be reposed with 3D models to add more variation in facial pose and shape [6,7]. GAN-based models have also been successfully deployed to generate hires synthetic face images [8] or edit visual attributes like age [9], lighting and pose [10], gender and expressions [11].…”

Section: Introductionmentioning

confidence: 99%

Synthesize & Learn: Jointly Optimizing Generative and Classifier Networks for Improved Drowsiness Detection

Banerjee

Joshi

Ghoneim

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Driving in a state of drowsiness is a major cause of road accidents, resulting in tremendous damage to life and property. Developing robust, automatic, real-time systems that can infer drowsiness states of drivers has the potential of making lifesaving impact. However, real-world drowsy driving datasets are unbalanced, due to the sparsity of drowsy driving events. We focus on the problem of alleviating the class imbalance problem by using generative adversarial networks (GAN) to synthesize examples of sparse classes directly in the featurespace. Our GAN-based framework simultaneously generates realistic examples of sparse classes while using the generated samples to improve the performance of a separate drowsiness classifier. We validate this approach in a real-world drowsiness dataset, where we demonstrate a classifier trained using this approach outperforms a stand-alone classifier trained without any GAN-based augmentations.

show abstract

“…One way to mitigate the imbalance problem, shown to work in multiple domains [56,8,13,98], is to introduce synthetic samples into the training set. Many approaches for generating synthetic data exist [16,7,39], none as successful as GANs [34] in generating realistic face images [14,47,48,24].…”

Section: Introductionmentioning

confidence: 99%

“…Instead of generating a single score for a distribution of synthetic images [105,42,76] or for image pairs [103,72], our goal is to infer an image-quality score on a continuum for a single synthetic face image. With this in mind, we run an Amazon Mechanical Turk (AMT) experiment where turkers are instructed to score the naturalness of synthetic face images, generated using different 3D-model [8] and GAN-based [47,48,75,2] synthesis approaches. We then build a feed forward CNN to learn representations from these images that map to their corresponding perceptual rating, using a margin based regression loss.…”

Section: Introductionmentioning

confidence: 99%

LEGAN: Disentangled Manipulation of Directional Lighting and Facial Expressions by Leveraging Human Perceptual Judgements

Banerjee,

Joshi,

Mahajan

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Building facial analysis systems that generalize to extreme variations in lighting and facial expressions is a challenging problem that can potentially be alleviated using natural-looking synthetic data. Towards that, we propose LEGAN, a novel synthesis framework that leverages perceptual quality judgments for jointly manipulating lighting and expressions in face images, without requiring paired training data. LEGAN disentangles the lighting and expression subspaces and performs transformations in the feature space before upscaling to the desired output image. The fidelity of the synthetic image is further refined by integrating a perceptual quality estimation model into the LEGAN framework as an auxiliary discriminator. The quality estimation model is learned from face images rendered using multiple synthesis methods and their crowd-sourced naturalness ratings using a margin-based regression loss. Using objective metrics like FID and LPIPS, LEGAN is shown to generate higher quality face images when compared with popular GAN models like pix2pix, CycleGAN and StarGAN for lighting and expression synthesis. We also conduct a perceptual study using images synthesized by LEGAN and other GAN models, trained with and without the quality based auxiliary discriminator, and show the correlation between our quality estimation and visual fidelity. Finally, we demonstrate the effectiveness of LEGAN as training data augmenter for expression recognition and face verification tasks.Preprint. Under review.

show abstract

Fast Face Image Synthesis With Minimal Training

Cited by 11 publications

References 64 publications

Driver Glance Classification In-the-wild: Towards Generalization Across Domains and Subjects

Driver Glance Classification In-the-wild: Towards Generalization Across Domains and Subjects

Synthesize & Learn: Jointly Optimizing Generative and Classifier Networks for Improved Drowsiness Detection

LEGAN: Disentangled Manipulation of Directional Lighting and Facial Expressions by Leveraging Human Perceptual Judgements

Contact Info

Product

Resources

About