Fairness in Deep Learning: A Computational Perspective

Du, Mengnan; Yang, Fan; Zou, Na; Hu, Xia

doi:10.1109/mis.2020.3000681

Cited by 154 publications

(100 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing fairness metrics evaluate the performance gap across different subgroups ( Du et al, 2020 ). However, these do not reflect the overall model performance.…”

Section: Methodsmentioning

confidence: 99%

“…Therefore, researchers have proposed several algorithms to mitigate the effect of bias on model prediction ( Alvi et al, 2018 ; Gong et al, 2019 ). However, there is generally a trade-off between fairness and model performance ( Du et al, 2019 ; Li and Vasconcelos, 2019 ). Removal of bias may affect the overall model performance while a high performing model may affect the performance of the under-represented subgroup.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction

et al. 2021

View full text Add to dashboard Cite

Modern deep learning systems have achieved unparalleled success and several applications have significantly benefited due to these technological advancements. However, these systems have also shown vulnerabilities with strong implications on the fairness and trustability of such systems. Among these vulnerabilities, bias has been an Achilles’ heel problem. Many applications such as face recognition and language translation have shown high levels of bias in the systems towards particular demographic sub-groups. Unbalanced representation of these sub-groups in the training data is one of the primary reasons of biased behavior. To address this important challenge, we propose a two-fold contribution: a bias estimation metric termed as Precise Subgroup Equivalence to jointly measure the bias in model prediction and the overall model performance. Secondly, we propose a novel bias mitigation algorithm which is inspired from adversarial perturbation and uses the PSE metric. The mitigation algorithm learns a single uniform perturbation termed as Subgroup Invariant Perturbation which is added to the input dataset to generate a transformed dataset. The transformed dataset, when given as input to the pre-trained model reduces the bias in model prediction. Multiple experiments performed on four publicly available face datasets showcase the effectiveness of the proposed algorithm for race and gender prediction.

show abstract

“…Existing fairness metrics evaluate the performance gap across different subgroups ( Du et al, 2020 ). However, these do not reflect the overall model performance.…”

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The unfairness of DNN can usually be classified into two groups from a computational perspective: discrimination in the result and inequality in the consistency of prediction [161]. Discrimination involves the phenomenon of unfavourable treatment of persons by DNN models because of the participation of certain ethnic groups.…”

Section: And Explainability: a Machine Learning Zoo Mini-tour And Explainable Ai A Review Of Machine Learning Interpretability Methods Pamentioning

confidence: 99%

On Deep Research Problems in Deep Learning

Ulhaq¹

2021

Preprint

View full text Add to dashboard Cite

The subject of deep learning has emerged in the last decade as one of the most promising approaches to machine learning. Today, certainly, much of the recent progress in artificial intelligence is due to it, but research challenges are still unresolved and remain open to the research community. This paper attempts to offer a comprehensive review of deep learning progress in active research frontiers. On the one side, by presenting a brief overview of deep learning success, we inspire researchers to work in deep learning. On the other hand, we examine a range of technical issues, and open research issues that we believe are relevant topics for exploratory research. As deep learning applies to various fields, we restrict this paper’s scope to visual recognition tasks to analyze these problems with a specific lens. However, these problems will be broadly applicable to other fields. It will make it easier for new researchers to recognize outstanding research problems in the deep learning domain.

show abstract

“…Besides accuracy, interpretability and fairness are two important aspects that businesses and researchers should take into consideration when designing, deploying, and maintaining machine learning models 54 . It is also well acknowledged that enhancing model interpretability is an important step towards developing fairer ML systems 55 since interpretations can help detecting and mitigating bias during data collection or labeling [56][57][58] . Given evaluation metrics from the two concepts, demonstrations of performance from different predictive models have been shown in literature to further investigate their interactions [59][60][61][62][63] .…”

Section: Interactions Between Interpretability and Fairnessmentioning

confidence: 99%

MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset

Meng

Trinh

et al. 2021

Preprint

View full text Add to dashboard Cite

The recent release of large-scale healthcare datasets has greatly propelled the research of data-driven deep learning models for healthcare applications. However, due to the nature of such deep black-boxed models, concerns about interpretability, fairness, and biases in healthcare scenarios where human lives are at stake call for a careful and thorough examination of both datasets and models. In this work, we focus on MIMIC-IV (Medical Information Mart for Intensive Care, version IV), the largest publicly available healthcare dataset, and conduct comprehensive analyses of dataset representation bias as well as interpretability and prediction fairness of deep learning models for in-hospital mortality prediction. In terms of interpretability, we observe that (1) the best-performing interpretability method successfully identifies critical features for mortality prediction on various prediction models; (2) demographic features are important for prediction. In terms of fairness, we observe that (1) there exists disparate treatment in prescribing mechanical ventilation among patient groups across ethnicity, gender and age; (2) all of the studied mortality predictors are generally fair while the IMV-LSTM (Interpretable Multi-Variable Long Short-Term Memory) model provides the most accurate and unbiased predictions across all protected groups. We further draw concrete connections between interpretability methods and fairness metrics by showing how feature importance from interpretability methods can be beneficial in quantifying potential disparities in mortality predictors.

show abstract

Fairness in Deep Learning: A Computational Perspective

Cited by 154 publications

References 35 publications

Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction

Subgroup Invariant Perturbation for Unbiased Pre-Trained Model Prediction

On Deep Research Problems in Deep Learning

MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset

Contact Info

Product

Resources

About