Measuring the Gender and Ethnicity Bias in Deep Models for Face Recognition

Acien, Alejandro; Morales, Aythami; Vera-Rodríguez, Rubén; Bartolome, Ivan; Fiérrez, Julián

doi:10.1007/978-3-030-13469-3_68

Cited by 42 publications

(37 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our study began with this question, which was investigated through the COVID-19 issue, the topic of greatest interest to people around the world. Recently, some studies on AI bias according to gender have been conducted mainly in image recognition and natural language processing fields ( Acien et al, 2018 ; Costa-jussà, 2019 ), but most AI studies have focused on improving performance through trial and error: hyperparameter search on networks ( Forghani, 2020 ). Especially with the prediction model of severity or mortality at an early stage of COVID-19 ( Altschul et al, 2020 ; Zhu et al, 2020 ; Lessmann et al, 2021 ; Paiva Proença Lobo Lopes et al, 2021 ; Shan et al, 2021 ; Yaşar et al, 2021 ), to our best knowledge, this paper serves as the first attempt to investigate the gender-specific models.…”

Section: Discussionmentioning

confidence: 99%

Gender Bias in Artificial Intelligence: Severity Prediction at an Early Stage of COVID-19

et al. 2021

View full text Add to dashboard Cite

Artificial intelligence (AI) technologies have been applied in various medical domains to predict patient outcomes with high accuracy. As AI becomes more widely adopted, the problem of model bias is increasingly apparent. In this study, we investigate the model bias that can occur when training a model using datasets for only one particular gender and aim to present new insights into the bias issue. For the investigation, we considered an AI model that predicts severity at an early stage based on the medical records of coronavirus disease (COVID-19) patients. For 5,601 confirmed COVID-19 patients, we used 37 medical records, namely, basic patient information, physical index, initial examination findings, clinical findings, comorbidity diseases, and general blood test results at an early stage. To investigate the gender-based AI model bias, we trained and evaluated two separate models—one that was trained using only the male group, and the other using only the female group. When the model trained by the male-group data was applied to the female testing data, the overall accuracy decreased—sensitivity from 0.93 to 0.86, specificity from 0.92 to 0.86, accuracy from 0.92 to 0.86, balanced accuracy from 0.93 to 0.86, and area under the curve (AUC) from 0.97 to 0.94. Similarly, when the model trained by the female-group data was applied to the male testing data, once again, the overall accuracy decreased—sensitivity from 0.97 to 0.90, specificity from 0.96 to 0.91, accuracy from 0.96 to 0.91, balanced accuracy from 0.96 to 0.90, and AUC from 0.97 to 0.95. Furthermore, when we evaluated each gender-dependent model with the test data from the same gender used for training, the resultant accuracy was also lower than that from the unbiased model.

show abstract

Section: Discussionmentioning

confidence: 99%

Gender Bias in Artificial Intelligence: Severity Prediction at an Early Stage of COVID-19

et al. 2021

View full text Add to dashboard Cite

show abstract

“…However, in order to be acceptable in a practical context, that performance level must be retained when implemented in a real-world context, using non-pristine images, and in a cost/form factor that is realistic for a store to deploy. Further, knowing that many machine-learning solutions are susceptible to degradation resulting from training dataset mismatch with respect to ethnicity [11], gender [12], image quality [13], lighting conditions [14], and combinations of these parameters with other characteristics [15][16][17], we chose to intentionally bombard the neural net model with different presentation attacks to quantify how quickly performance degrades.…”

Section: Motivationmentioning

confidence: 99%

Real-Time Mask Recognition

Billings

Michaels

2021

IoT

View full text Add to dashboard Cite

While a variety of image processing studies have been performed to quantify the potential performance of neural network-based models using high-quality still images, relatively few studies seek to apply those models to a real-time operational context. This paper seeks to extend prior work in neural-network-based mask detection algorithms to a real-time, low-power deployable context that is conducive to immediate installation and use. Particularly relevant in the COVID-19 era with varying rules on mask mandates, this work applies two neural network models to inference of mask detection in both live (mobile) and recorded scenarios. Furthermore, an experimental dataset was collected where individuals were encouraged to use presentation attacks against the algorithm to quantify how perturbations negatively impact model performance. The results from evaluation on the experimental dataset are further investigated to identify the degradation caused by poor lighting and image quality, as well as to test for biases within certain demographics such as gender and ethnicity. In aggregate, this work validates the immediate feasibility of a low-power and low-cost real-time mask recognition system.

show abstract

“…Other recent works in face recognition (FR) technology introduce additional modalities, such as profile information, to the problem of bias [38], [39]. Other questions concern the measures of biases in FR systems [28], [33], FR templates [40], score level biases [27], and biases in existing (i.e., trained) models [41]. Wang et al introduced a reinforcement learning-based race balanced network to find optimal margins for non-Caucasians as a Markov decision process before passing to the deep model that learns policies for an agent to select margins as an approximation of the Q-value function-it reduces variations in scattering between features across races [42].…”

Section: Bias In Frmentioning

confidence: 99%

“…We train a multi-layered perceptron (MLP) to classify subgroups on top of the features to show the extent that the proposed removes subgroup information. We can then measure the amount of information present in the face representation [33].…”

Section: Privacy Preserving Experimentsmentioning

confidence: 99%

“…T HE more our society integrates with machine learning (ML), the higher the interest in topics such as bias, fairness, and the implications of using the technology [1]- [3]. And, the more we depend on technology that speeds up or automates everyday tasks, the more attention concepts such as biased and unfair algorithms should receive [4], [5].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Balancing Biases and Preserving Privacy on Balanced Faces in the Wild

Robinson¹,

Qin²,

Henon³

et al. 2021

Preprint

View full text Add to dashboard Cite

There are demographic biases in current models used for facial recognition (FR). Our Balanced Faces In the Wild (BFW) dataset serves as a proxy to measure bias across ethnicity and gender subgroups, allowing one to characterize FR performances per subgroup. We show performances are non-optimal when a single score threshold is used to determine whether sample pairs are genuine or imposter. Across subgroups, performance ratings vary from the reported across the entire dataset. Thus, claims of specific error rates only hold true for populations matching that of the validation data. We mitigate the imbalanced performances using a novel domain adaptation learning scheme on the facial features extracted using state-of-the-art. Not only does this technique balance performance, but it also boosts the overall performance. A benefit of the proposed is to preserve identity information in facial features while removing demographic knowledge in the lower dimensional features. The removal of demographic knowledge prevents future potential biases from being injected into decision-making. This removal satisfies privacy concerns. We explore why this works qualitatively; we also show quantitatively that subgroup classifiers can no longer learn from the features mapped by the proposed.

show abstract

Measuring the Gender and Ethnicity Bias in Deep Models for Face Recognition

Cited by 42 publications

References 16 publications

Gender Bias in Artificial Intelligence: Severity Prediction at an Early Stage of COVID-19

Gender Bias in Artificial Intelligence: Severity Prediction at an Early Stage of COVID-19

Real-Time Mask Recognition

Balancing Biases and Preserving Privacy on Balanced Faces in the Wild

Contact Info

Product

Resources

About