Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction

Abstract

Modern deep learning systems have achieved unparalleled success and several applications have significantly benefited due to these technological advancements. However, these systems have also shown vulnerabilities with strong implications on the fairness and trustability of such systems. Among these vulnerabilities, bias has been an Achilles’ heel problem. Many applications such as face recognition and language translation have shown high levels of bias in the systems towards particular demographic sub-groups. Unbalanced representation of these sub-groups in the training data is one of the primary reasons of biased behavior. To address this important challenge, we propose a two-fold contribution: a bias estimation metric termed as Precise Subgroup Equivalence to jointly measure the bias in model prediction and the overall model performance. Secondly, we propose a novel bias mitigation algorithm which is inspired from adversarial perturbation and uses the PSE metric. The mitigation algorithm learns a single uniform perturbation termed as Subgroup Invariant Perturbation which is added to the input dataset to generate a transformed dataset. The transformed dataset, when given as input to the pre-trained model reduces the bias in model prediction. Multiple experiments performed on four publicly available face datasets showcase the effectiveness of the proposed algorithm for race and gender prediction.

Anatomizing Bias in Facial Analysis

“…They proposed to adaptively select mini-batches for improving model fairness. A bias mitigation algorithm based on adversarial perturbation is proposed by (Majumdar et al 2020). The proposed algorithm learns a subgroup invariant perturbation to be added to the input database to generate a transformed database.…”

“…The focus of researchers is towards understanding bias in model prediction and mitigating its effect to obtain unbiased outcomes (Ntoutsi et 2020). In this regard, multiple analyses have been performed to detect the sources of bias (Celis and Rao 2019;Krishnapriya et al 2020), databases are proposed to understand the effect of bias (Wang et al 2019a;Karkkainen and Joo 2021), and algorithms are developed to mitigate the effect of bias in model predictions (Majumdar et al 2020;Joo and Kärkkäinen 2020).…”

Unravelling the Effect of Image Distortions for Biased Prediction of Pre-trained Face Recognition Models

“…In the recent literature, several studies have focused on detection and mitigation of bias present in deep learning models [12,24,37]. However, very few focus on the analysis of this prevalent bias.…”

On Learning Deep Models with Imbalanced Data Distribution

“…Currently, we are working on proposing a metric to jointly measure bias in model prediction and the overall model performance. Additionally, we aim to use this metric in the bias mitigation algorithm for performance enhancement (Majumdar et al 2020).…”