Maintaining fairness across distribution shift: do we have viable solutions for real-world applications?

Schrouff, Jessica; Harris, Natalie; Koyejo, Oluwasanmi; Alabdulmohsin, Ibrahim; Schnider, Eva; Opsahl-Ong, Krista; Brown, Alex; Roy, Subhrajit; Mincu, Diana; Chen, Christina; Dieng, Awa; Liu, Yuan; Natarajan, Vivek; Karthikesalingam, Alan; Heller, Katherine; Chiappa, Silvia; D’Amour, Alexander

doi:10.48550/arxiv.2202.01034

Cited by 8 publications

(8 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gianfrancesco et al (2018) performed a similar analysis for models operating on electronic health records. When evaluating machine learning systems in terms of certain fairness criteria, it is important to keep in mind that ensuring fairness in a source domain does not guarantee fairness in a different target domain under significant distribution shifts (Schrouff et al, 2022). Last but not least, there are multiple definitions of fairness in recent literature and different fairness metrics are often at odds with each other as noted by Ricci Lara et al (2022).…”

Section: Histopathologymentioning

confidence: 99%

Generative models improve fairness of medical classifiers under distribution shifts

Wiles

Albuquerque

Rebuffi

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Domain generalization remains a ubiquitous challenge for machine learning in healthcare. Model performance in real-world conditions might be lower than expected due to discrepancies between the data encountered in deployment environments and datasets used for model development. Under-representation of some groups or conditions during model development is a common cause of this phenomenon, which can have serious implications as it can exacerbate bias against groups, individuals or conditions and propagate unintended harms in their care. This challenge is often not readily addressed by targeted data acquisition and “labelling” by expert clinicians, which can be prohibitively expensive or practically impossible due to the rarity of diseases, conditions, or available clinical expertise. We hypothesize that advances in generative artificial intelligence may help mitigate this unmet need in a steerable fashion, algorithmically enriching our training dataset with synthetic examples that address shortfalls of underrepresented conditions or subgroups. We show that generative models can automatically learn realistic augmentations from data in a label-efficient manner. In particular, we leverage the higher abundance of unlabelled data to model the underlying distribution of different conditions and subgroups for an imaging modality. By conditioning generative models on appropriate labels (e.g., diagnostic labels and / or sensitive attribute labels), we can steer the distribution of synthetic examples according to specific requirements. We demonstrate that these learned augmentations make models more robust and statistically fair in- and out-of-distribution. To evaluate the generality of our approach, we study three distinct medical imaging contexts of varying difficulty: (i) histopathology images from a publicly available and widely adopted generalization benchmark, (ii) chest X-rays from publicly available clinical datasets, and (iii) dermatology images characterized by complex shifts and imaging conditions. The latter constitutes a particularly unstructured domain with various challenges. Two of these imaging modalities further require operating at a high-resolution, which requires developing faithful super-resolution techniques to recover fine details of each health condition. Complementing real training samples with synthetic ones improves the robustness of models in all three medical tasks and increases fairness by improving the accuracy of clinical diagnosis within underrepresented groups. Our proposed approach leads to stark improvements out-of-distribution across modalities: 7.7% prediction accuracy improvement in histopathology, 5.2% in chest radiology with 44.6% lower fairness gap and a striking 63.5% improvement in high-risk sensitivity for dermatology with a 7.5x reduction in fairness gap.

show abstract

Section: Histopathologymentioning

confidence: 99%

Generative models improve fairness of medical classifiers under distribution shifts

Wiles

Albuquerque

Rebuffi

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The degraded performance of fair models under a distribution shift would trigger new bias and discrimination issues. There are some works [Rezaei et al, 2021, Schrouff et al, 2022, Singh et al, 2021, Giguere et al, 2022] that aim to solve fairness under various distribution shifts. For example, Rezaei et al [2021] explore the fairness under covariate shift, where the inputs change while the label distribution is in-distribution.…”

Section: Related Workmentioning

confidence: 99%

Weight Perturbation Can Help Fairness under Distribution Shift

Jiang¹,

Han²,

Jin³

et al. 2023

Preprint

View full text Add to dashboard Cite

Fairness in machine learning has attracted increasing attention in recent years. The fairness methods improving algorithmic fairness for in-distribution data may not perform well under distribution shift. In this paper, we first theoretically demonstrate the inherent connection between distribution shift, data perturbation, and weight perturbation. Subsequently, we analyze the sufficient conditions to guarantee fairness (i.e., low demographic parity) for the target dataset, including fairness for the source dataset, and low prediction difference between the source and target dataset for each sensitive attribute group. Motivated by these sufficient conditions, we propose robust fairness regularization (RFR) by considering the worst case within the weight perturbation ball for each sensitive attribute group. In this way, the maximization problem can be simplified as two forward and two backward propagations for each update of model parameters. We evaluate the effectiveness of our proposed RFR algorithm on synthetic and real distribution shifts across various datasets. Experimental results demonstrate that RFR achieves better fairness-accuracy trade-off performance compared with several baselines.

show abstract

“…Schrouff et al [163] evaluates how realistic the assumptions made by Singh et al [162] as well as other works are [80,164]. They categorize distribution shifts into four categories: demographic shift, covariate shift, label shift, and compound shift, independently of the causal graph considered.…”

Section: Fairness Under Distribution Shiftsmentioning

confidence: 99%

Causal Machine Learning: A Survey and Open Problems

Jean¹,

Lynch²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Causal Machine Learning (CausalML) is an umbrella term for machine learning methods that formalize the data-generation process as a structural causal model (SCM). This allows one to reason about the effects of changes to this process (i.e., interventions) and what would have happened in hindsight (i.e., counterfactuals). We categorize work in CausalML into five groups according to the problems they tackle: (1) causal supervised learning, (2) causal generative modeling, (3) causal explanations, (4) causal fairness, (5) causal reinforcement learning. For each category, we systematically compare its methods and point out open problems. Further, we review modality-specific applications in computer vision, natural language processing, and graph representation learning. Finally, we provide an overview of causal benchmarks and a critical discussion of the state of this nascent field, including recommendations for future work.

show abstract

Maintaining fairness across distribution shift: do we have viable solutions for real-world applications?

Cited by 8 publications

References 41 publications

Generative models improve fairness of medical classifiers under distribution shifts

Generative models improve fairness of medical classifiers under distribution shifts

Weight Perturbation Can Help Fairness under Distribution Shift

Causal Machine Learning: A Survey and Open Problems

Contact Info

Product

Resources

About