Ethical limitations of algorithmic fairness solutions in health care machine learning

McCradden, Melissa D; Joshi, Shalmali; Mazwi, Mjaye; Anderson, James A.

doi:10.1016/s2589-7500(20)30065-0

Cited by 153 publications

(105 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is imperative to acknowledge that data sources and algorithms represent just one dimension of bias, because overreliance on technical metrics can bring unexpected consequences. 13 Prediction algorithms almost always intend to influence the clinical decision-making process, and the biases of those developing and using them would have greater impact on whether they ultimately perpetuate inequality. 30,31 For example, a clinician may recognize the potential disparity in diagnosis and intentionally compensate for the algorithmic bias.…”

Section: Discussionmentioning

confidence: 99%

“…[8][9][10][11] However, gaps between such advances and solutions for algorithmic bias in clinical prediction persist because of technical difficulties, complexities of high dimensional health data, lack of knowledge of underlying causal structures, and challenges to algorithm appraisal. 12,13 Few examples in health care to date use methods to reduce bias. Influential work by Obermeyer et al 7 suggests a relabeling approach, replacing erroneous or biased target outcomes of machine learning models with alternatives.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum Depression

Park

Singh

et al. 2021

JAMA Netw Open

View full text Add to dashboard Cite

IMPORTANCEThe lack of standards in methods to reduce bias for clinical algorithms presents various challenges in providing reliable predictions and in addressing health disparities. OBJECTIVE To evaluate approaches for reducing bias in machine learning models using a real-world clinical scenario. DESIGN, SETTING, AND PARTICIPANTSHealth data for this cohort study were obtained from the IBM MarketScan Medicaid Database. Eligibility criteria were as follows: (1) Female individuals aged 12 to 55 years with a live birth record identified by delivery-related codes from January 1, 2014, through December 31, 2018; (2) greater than 80% enrollment through pregnancy to 60 days post partum; and (3) evidence of coverage for depression screening and mental health services. Statistical analysis was performed in 2020. EXPOSURES Binarized race (Black individuals and White individuals). MAIN OUTCOMES AND MEASURES Machine learning models (logistic regression [LR], randomforest, and extreme gradient boosting) were trained for 2 binary outcomes: postpartum depression (PPD) and postpartum mental health service utilization. Risk-adjusted generalized linear models were used for each outcome to assess potential disparity in the cohort associated with binarized race (Black or White). Methods for reducing bias, including reweighing, Prejudice Remover, and removing race from the models, were examined by analyzing changes in fairness metrics compared with the base models. Baseline characteristics of female individuals at the top-predicted risk decile were compared for systematic differences. Fairness metrics of disparate impact (DI, 1 indicates fairness) and equal opportunity difference (EOD, 0 indicates fairness). RESULTS Among 573 634 female individuals initially examined for this study, 314 903 were White (54.9%), 217 899 were Black (38.0%), and the mean (SD) age was 26.1 (5.5) years. The risk-adjusted odds ratio comparing White participants with Black participants was 2.06 (95% CI, 2.02-2.10) for clinically recognized PPD and 1.37 (95% CI, 1.33-1.40) for postpartum mental health service utilization. Taking the LR model for PPD prediction as an example, reweighing reduced bias as measured by improved DI and EOD metrics from 0.31 and −0.19 to 0.79 and 0.02, respectively.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Comparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum Depression

Park

Singh

et al. 2021

JAMA Netw Open

View full text Add to dashboard Cite

show abstract

“…Thus, it is essential to put humans in the loop for accountability in decisions that affect patient care. 52,53 A social concern is the impact of AI on patient-provider relationships. The human touch, empathy, understanding, and judgment are critical components of healing and patient care.…”

Section: Trustmentioning

confidence: 99%

A Proposed Framework on Integrating Health Equity and Racial Justice into the Artificial Intelligence Development Lifecycle

Dankwa‐Mullan¹,

Scheufele²,

Matheny³

et al. 2021

Journal of Health Care for the Poor and Underserved

View full text Add to dashboard Cite

“…and others use human actors such as nursing staff, whose activities cannot change quickly (e.g., process changes, policy implications, etc., take time to change). This is often reflected in model or algorithmic bias or fairness when these biases get embedded in digital services [ 72 ]. For example, the algorithms used to make predictions on heart conditions may be biased if they undercount women and especially women of color, impacting diagnoses and treatment plans [ 73 ].…”

Section: Designing a Digital Platform To Support Agilitymentioning

confidence: 99%

An Agile Digital Platform to Support Population Health—A Case Study of a Digital Platform to Support Patients with Delirium Using IoT, NLP, and AI

Tanniru

Agarwal

Sokan

et al. 2021

IJERPH

View full text Add to dashboard Cite

For an organization to be customer centric and service oriented requires that it use each encounter with a customer to create value, leverage advanced technologies to design digital services to fulfill the value, and assess perceived value-in-use to continue to revise the value as customer expectations evolve. The adaptation of value cycles to address the rapid changes in customer expectations requires agile digital platforms with dynamic software ecosystems interacting with multiple actors. For public health agencies focused on population health, these agile digital platforms should provide tailored care to address the distinct needs of select population groups. Using prior research on aging and dynamic software ecosystems, this paper develops a template for the design of an agile digital platform to support value cycle activities among clinical and non-clinical actors, including population groups. It illustrates the design of an agile digital platform to support clients that suffer from delirium, using digital services that leverage Internet of Things, natural language processing, and AI that uses real-time data for learning and care adaption. We conclude the paper with directions for future research.

show abstract

Ethical limitations of algorithmic fairness solutions in health care machine learning

Cited by 153 publications

References 9 publications

Comparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum Depression

Comparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum Depression

A Proposed Framework on Integrating Health Equity and Racial Justice into the Artificial Intelligence Development Lifecycle

An Agile Digital Platform to Support Population Health—A Case Study of a Digital Platform to Support Patients with Delirium Using IoT, NLP, and AI

Contact Info

Product

Resources

About