Gender Privacy: An Ensemble of Semi Adversarial Networks for Confounding Arbitrary Gender Classifiers

Abstract

Recent research has proposed the use of Semi Adversarial Networks (SAN) for imparting privacy to face images. SANs are convolutional autoencoders that perturb face images such that the perturbed images cannot be reliably used by an attribute classifier (e.g., a gender classifier) but can still be used by a face matcher for matching purposes. However, the generalizability of SANs across multiple arbitrary gender classifiers has not been demonstrated in the literature. In this work, we tackle the generalization issue by designing an ensemble SAN model that generates a diverse set of perturbed outputs for a given input face image. This is accomplished by enforcing diversity among the individual models in the ensemble through the use of different data augmentation techniques. The goal is to ensure that at least one of the perturbed output faces will confound an arbitrary, previously unseen gender classifier. Extensive experiments using different unseen gender classifiers and face matchers are performed to demonstrate the efficacy of the proposed paradigm in imparting gender privacy to face images.

Some Research Problems in Biometrics: The Future Beckons

“…Recent research has explored the notion of controllable privacy [29] where specific ancillary cues are suppressed in the raw image ( Figure 11). For example, semi-adversarial neural networks have been designed to remove gender cues from a face image, through a series of perturbations, such that the performance of automated gender classifiers is confounded but the performance of face matchers is retained [20]. Introduction of the EU General Data Protection Regulation (GDPR) has reinforced the importance of designing privacy-preserving methods in the context of biometric systems.…”

FlowSAN: Privacy-Enhancing Semi-Adversarial Networks to Confound Arbitrary Face-Based Gender Classifiers

“…Protocol (a) was adapted from [45] and is further described in Section 3.1. For evaluating models trained in the ensemble, we applied two techniques: 1) taking the average output from SAN models which we denote as Ens-Avg, and 2) randomly selecting the output which we denote as Ens-Gibbs.…”

“…The SAN model was shown to be able to derive perturbations that are transferable to two unseen gender classifiers. In [45], we investigated the generalizability of SAN models across multiple arbitrary gender classifiers and formulated an ensemble SAN model with a training scheme based on different data augmentation techniques, to enhance diversity in the ensemble of SAN models. Furthermore, we explored the effectiveness of randomly selecting a perturbed image from an ensemble of SAN models, which we refer to as Ens-Gibbs [45].…”

“…In our previous work [45], we proposed an ensemble approach for generalizing SAN models to unseen gender classifiers. The objective of an ensemble SAN was to create n SAN models such that their union can span a larger subset of the hypothesis space compared to a single SAN model.…”

“…In our previous work [44], we empirically showed that the ability to predict gender information, using an unseen gender classifier from outputs of the SAN model, is successfully diminished. In [45], we defined the generalizability of the SAN model as its ability to confound arbitrary unseen 1 1. The term "unseen" indicates that a certain classifier (or face matcher) was not used during the training stage.…”