Addressing bias in prediction models by improving subpopulation calibration

Abstract

Objective To illustrate the problem of subpopulation miscalibration, to adapt an algorithm for recalibration of the predictions, and to validate its performance. Materials and Methods In this retrospective cohort study, we evaluated the calibration of predictions based on the Pooled Cohort Equations (PCE) and the fracture risk assessment tool (FRAX) in the overall population and in subpopulations defined by the intersection of age, sex, ethnicity, socioeconomic status, and immigration history. We next applied the recalibration algorithm and assessed the change in calibration metrics, including calibration-in-the-large. Results 1 021 041 patients were included in the PCE population, and 1 116 324 patients were included in the FRAX population. Baseline overall model calibration of the 2 tested models was good, but calibration in a substantial portion of the subpopulations was poor. After applying the algorithm, subpopulation calibration statistics were greatly improved, with the variance of the calibration-in-the-large values across all subpopulations reduced by 98.8% and 94.3% in the PCE and FRAX models, respectively. Discussion Prediction models in medicine are increasingly common. Calibration, the agreement between predicted and observed risks, is commonly poor for subpopulations that were underrepresented in the development set of the models, resulting in bias and reduced performance for these subpopulations. In this work, we empirically evaluated an adapted version of the fairness algorithm designed by Hebert-Johnson et al. (2017) and demonstrated its use in improving subpopulation miscalibration. Conclusion A postprocessing and model-independent fairness algorithm for recalibration of predictive models greatly decreases the bias of subpopulation miscalibration and thus increases fairness and equality.

Generalizability Challenges of Mortality Risk Prediction Models: A Retrospective Analysis on a Multi-center Database

“…Differences in these two metrics have been employed in recent studies for bias analysis. 16,17 FNR quantifies the rate at which patients with the observed outcome of death were misclassified. Thus, a high FNR for the score may lead to an increase in undertreatment, and high DisparityFNR (in absolute value) highlights large differences in such undertreatment across groups.…”

“…Hence, in assessment of generalizability, we add another set of measures to our analyses which we refer to as "fairness" metrics, following the algorithmic fairness literature. [15][16][17] Such performance checks are important, especially given evidence on racial bias in medical decision-making tools. 4,13,14 The primary objective of this study is to evaluate the external validity of predictive models for clinical decision making across hospitals and geographies in terms of the metrics -predictive discrimination (area under the receiver operating characteristic curve), calibration (calibration slope), 18 and algorithmic fairness (disparity in false negative rates and disparity in calibration slopes).…”

MIMIC-IF: Interpretability and Fairness Evaluation of Deep Learning Models on MIMIC-IV Dataset

“…Fairness has been defined in various ways considering different contexts or applications, two of them are the most widely leveraged for bias detection and correction: Equal Opportunity, where the predictions are required to have equal true positive rate across two demographics, and Equalized Odds, where an additional constraint is put on the predictor to have equal false positive rate 43 . To derive fair decisions with machine learning algorithms, three categories of approaches have been proposed to mitigate biases 42,44 : 1)Pre-processing: the original dataset is transformed so that the underlying discrimination towards some groups is removed 45 ; 2) In-processing: either by adding a penalization term in the objective function 46 or imposing a fairness-relevant constraint 47 ; 3) Post-processing: further recompute the results from predictors to improve fairness 48 .…”

Low adherence to existing model reporting guidelines by commonly used clinical prediction models

“…5 Some potential causes of this chasm are that current models are not useful, 4,6,7 reliable, 8,9 or fair. [10][11][12][13][14][15][16][17][18] Nevertheless, predictive models have been deployed in healthcare settings without transparency or independent validation, 19,20 and their subsequent failures have been met with public outcry. 2,[21][22][23] Adhering to model reporting guidelines is one way to improve the usefulness, [24][25][26][27][28] fairness, 29,30 and reliability 27,[31][32][33][34] of clinical predictive models.…”