FlowSAN: Privacy-Enhancing Semi-Adversarial Networks to Confound Arbitrary Face-Based Gender Classifiers

Mirjalili, Vahid; Raschka, Sebastian; Ross, Arun

doi:10.1109/access.2019.2924619

Cited by 44 publications

(53 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [25]- [27], Mirjalili et al proposed semi-adversarial networks consisting of a convolutional autoencoder, a gender classifier, and a face matcher. It enhances the soft-biometric privacy on image level.…”

Section: Related Workmentioning

confidence: 99%

“…For the evaluation, we consider function creep attacks to the privacy-sensitive attribute gender as done in previous works [2], [25]- [27], [29], [46], [47]. The reason for this choice is that gender information can be estimated from face templates with very high accuracies [48], [49].…”

Section: Function Creep Attacksmentioning

confidence: 99%

“…It can be seen that all approaches show a degraded face verification performance compared to the original embeddings. This is shown in every privacy-enhancing work [27], [29], [45], [46], since soft-biometric privacy defines a trade-off between maintaining identity information and suppressing privacy-sensitive attributes. For lower FMR, NFR [47] is an exception of this trade-off.…”

Section: A Face Verification Performancementioning

confidence: 99%

See 2 more Smart Citations

PE-MIU: A Training-Free Privacy-Enhancing Face Recognition Approach Based on Minimum Information Units

et al. 2020

View full text Add to dashboard Cite

Research on soft-biometrics showed that privacy-sensitive information can be deduced from biometric data. Utilizing biometric templates only, information about a persons gender, age, ethnicity, sexual orientation, and health state can be deduced. For many applications, these templates are expected to be used for recognition purposes only. Thus, extracting this information raises major privacy issues. Previous work proposed two kinds of learning-based solutions for this problem. The first ones provide strong privacy-enhancements, but limited to pre-defined attributes. The second ones achieve more comprehensive but weaker privacy-improvements. In this work, we propose a Privacy-Enhancing face recognition approach based on Minimum Information Units (PE-MIU). PE-MIU, as we demonstrate in this work, is a privacy-enhancement approach for face recognition templates that achieves strong privacy-improvements and is not limited to pre-defined attributes. We exploit the structural differences between face recognition and facial attribute estimation by creating templates in a mixed representation of minimal information units. These representations contain pattern of privacy-sensitive attributes in a highly randomized form. Therefore, the estimation of these attributes becomes hard for function creep attacks. During verification, these units of a probe template are assigned to the units of a reference template by solving an optimal best-matching problem. This allows our approach to maintain a high recognition ability. The experiments are conducted on three publicly available datasets and with five state-of-the-art approaches. Moreover, we conduct the experiments simulating an attacker that knows and adapts to the systems privacy mechanism. The experiments demonstrate that PE-MIU is able to suppress privacy-sensitive information to a significantly higher degree than previous work in all investigated scenarios. At the same time, our solution is able to achieve a verification performance close to that of the unmodified recognition system. Unlike previous works, our approach offers a strong and comprehensive privacy-enhancement without the need of training.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Function Creep Attacksmentioning

confidence: 99%

Section: A Face Verification Performancementioning

confidence: 99%

See 1 more Smart Citation

PE-MIU: A Training-Free Privacy-Enhancing Face Recognition Approach Based on Minimum Information Units

et al. 2020

View full text Add to dashboard Cite

show abstract

“…A considerable amount of research has been done over recent years to address the privacy-related issues, needs, and legislative requirements (associated with biometric data) discussed above, including work on imaging sensors with built-in privacy protection [5], [6], deidentification 1. Accessible from: https://gdpr-info.eu/ techniques for biometric data [7], [8], [9], [10], adversarial approaches capable of confounding (automatic) recognition techniques [11], [12], [13], schemes that allow for privacypreserving data sharing [14], [15], template protection techniques [16], cancelable biometrics [17], [18], and others. A significant portion of this work share a common characteristic in that they try to mitigate privacy concerns by reducing the biometric utility of the captured data.…”

Section: Introductionmentioning

confidence: 99%

Privacy–Enhancing Face Biometrics: A Comprehensive Survey

Meden

Rot

Terhörst

et al. 2021

IEEE Trans.Inform.Forensic Secur.

Self Cite

108

View full text Add to dashboard Cite

Biometric recognition technology has made significant advances over the last decade and is now used across a number of services and applications. However, this widespread deployment has also resulted in privacy concerns and evolving societal expectations about the appropriate use of the technology. For example, the ability to automatically extract age, gender, race, and health cues from biometric data has heightened concerns about privacy leakage. Face recognition technology, in particular, has been in the spotlight, and is now seen by many as posing a considerable risk to personal privacy. In response to these and similar concerns, researchers have intensified efforts towards developing techniques and computational models capable of ensuring privacy to individuals, while still facilitating the utility of face recognition technology in several application scenarios. These efforts have resulted in a multitude of privacy-enhancing techniques that aim at addressing privacy risks originating from biometric systems and providing technological solutions for legislative requirements set forth in privacy laws and regulations, such as GDPR. The goal of this overview paper is to provide a comprehensive introduction into privacy-related research in the area of biometrics and review existing work on Biometric Privacy-Enhancing Techniques (B-PETs) applied to face biometrics. To make this work useful for as wide of an audience as possible, several key topics are covered as well, including evaluation strategies used with B-PETs, existing datasets, relevant standards, and regulations and critical open issues that will have to be addressed in the future.

show abstract

“…First, for many applications, the users do only permit to have access to the information related to recognition [32] and extracting additional information without a person's consent is considered a violation of their privacy [24]. This is known as soft-biometric privacy [32] and solutions are either build on image- [30], [31], [34] or embedding-level [5], [42], [45], [51]. Second, the attributes stored in biometric face embeddings can indicate biased performances related to these attributes that might result in unfair performance differences.…”

Section: Introductionmentioning

confidence: 99%

On Soft-Biometric Information Stored in Biometric Face Embeddings

Terhörst

Fährmann

Damer

et al. 2021

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

The success of modern face recognition systems is based on the advances of deeply-learned features. These embeddings aim to encode the identity of an individual such that these can be used for recognition. However, recent works have shown that more information beyond the user's identity is stored in these embeddings, such as demographics, image characteristics, and social traits. This raises privacy and bias concerns in face recognition. We investigate the predictability of 73 different soft-biometric attributes on three popular face embeddings with different learning principles. The experiments were conducted on two publicly available databases. For the evaluation, we trained a massive attribute classifier such that can accurately state the confidence of its predictions. This enables us to derive more sophisticated statements about the attribute predictability. The results demonstrate that the majority of the investigated attributes are encoded in face embeddings. For instance, a strong encoding was found for demographics, haircolors, hairstyles, beards, and accessories. Although face recognition embeddings are trained to be robust against non-permanent factors, we found that specifically these attributes are easily-predictable from face embeddings. We hope our findings will guide future works to develop more privacy-preserving and bias-mitigating face recognition technologies.

show abstract

FlowSAN: Privacy-Enhancing Semi-Adversarial Networks to Confound Arbitrary Face-Based Gender Classifiers

Cited by 44 publications

References 51 publications

PE-MIU: A Training-Free Privacy-Enhancing Face Recognition Approach Based on Minimum Information Units

PE-MIU: A Training-Free Privacy-Enhancing Face Recognition Approach Based on Minimum Information Units

Privacy–Enhancing Face Biometrics: A Comprehensive Survey

On Soft-Biometric Information Stored in Biometric Face Embeddings

Contact Info

Product

Resources

About