Algorithmic fairness in artificial intelligence for medicine and healthcare

Chen, Richard J.; Wang, Judy J.; Williamson, Drew F. K.; Chen, Tiffany; Lipková, Jana; Lu, Ming; Sahai, Sharifa; Mahmood, Faisal

doi:10.1038/s41551-023-01056-8

Cited by 62 publications

(29 citation statements)

References 274 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several measures have been proposed, focusing on different aspects of the prediction. For example, while demographic parity aims to match the proportion of positive predictions across subgroups, equalized odds intends to homogenize both true-positive and false-positive rates . However, there is currently a lack of standard methods for measuring and mitigating discrimination in ML models …”

Section: Discussionmentioning

confidence: 99%

“…For example, while demographic parity aims to match the proportion of positive predictions across subgroups, equalized odds intends to homogenize both true-positive and false-positive rates. 29 However, there is currently a lack of standard methods for measuring and mitigating discrimination in ML models. [30][31][32] When assessing fairness, it is important to consider race and ethnicity beyond binary comparisons between privileged and unprivileged groups.…”

Section: Jama Network Open | Public Healthmentioning

confidence: 99%

See 1 more Smart Citation

Fairness of Machine Learning Algorithms for Predicting Foregone Preventive Dental Care for Adults

Schuch,

Furtado,

Silva

et al. 2023

JAMA Netw Open

View full text Add to dashboard Cite

ImportanceAccess to routine dental care prevents advanced dental disease and improves oral and overall health. Identifying individuals at risk of foregoing preventive dental care can direct prevention efforts toward high-risk populations.ObjectiveTo predict foregone preventive dental care among adults overall and in sociodemographic subgroups and to assess the algorithmic fairness.Design, Setting, and ParticipantsThis prognostic study was a secondary analyses of longitudinal data from the US Medical Expenditure Panel Survey (MEPS) from 2016 to 2019, each with 2 years of follow-up. Participants included adults aged 18 years and older. Data analysis was performed from December 2022 to June 2023.ExposureA total of 50 predictors, including demographic and socioeconomic characteristics, health conditions, behaviors, and health services use, were assessed.Main Outcomes and MeasuresThe outcome of interest was foregoing preventive dental care, defined as either cleaning, general examination, or an appointment with the dental hygienist, in the past year.ResultsAmong 32 234 participants, the mean (SD) age was 48.5 (18.2) years and 17 386 participants (53.9%) were female; 1935 participants (6.0%) were Asian, 5138 participants (15.9%) were Black, 7681 participants (23.8%) were Hispanic, 16 503 participants (51.2%) were White, and 977 participants (3.0%) identified as other (eg, American Indian and Alaska Native) or multiple racial or ethnic groups. There were 21 083 (65.4%) individuals who missed preventive dental care in the past year. The algorithms demonstrated high performance, achieving an area under the receiver operating characteristic curve (AUC) of 0.84 (95% CI, 0.84-0.85) in the overall population. While the full sample model performed similarly when applied to White individuals and older adults (AUC, 0.88; 95% CI, 0.87-0.90), there was a loss of performance for other subgroups. Removing the subgroup-sensitive predictors (ie, race and ethnicity, age, and income) did not impact model performance. Models stratified by race and ethnicity performed similarly or worse than the full model for all groups, with the lowest performance for individuals who identified as other or multiple racial groups (AUC, 0.76; 95% CI, 0.70-0.81). Previous pattern of dental visits, health care utilization, dental benefits, and sociodemographic characteristics were the highest contributing predictors to the models’ performance.Conclusions and RelevanceFindings of this prognostic study using cohort data suggest that tree-based ensemble machine learning models could accurately predict adults at risk of foregoing preventive dental care and demonstrated bias against underrepresented sociodemographic groups. These results highlight the importance of evaluating model fairness during development and testing to avoid exacerbating existing biases.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Jama Network Open | Public Healthmentioning

confidence: 99%

Fairness of Machine Learning Algorithms for Predicting Foregone Preventive Dental Care for Adults

Schuch,

Furtado,

Silva

et al. 2023

JAMA Netw Open

View full text Add to dashboard Cite

show abstract

“…These characteristics enable AI-based clinical decision support systems to help providers overcome inherent constraints of bounded rationality (Camerer, 2018;Rawson et al, 2019). We acknowledge that by virtue of being trained using humangenerated data, AI might introduce pre-existing human bias into AI-informed decision-making (Chen, Chen, et al, 2021;Rajkomar et al, 2018). Underscoring this point, Obermeyer et al (2019) found evidence for racial bias in a widely used, commercially available algorithm where the health risks of Black patients were systematically underestimated compared to White patients.…”

Section: Susceptibility To Cognitive and Social Biasesmentioning

confidence: 99%

“…Therefore, it is not hard to see why physicians may be reluctant to allow such decisions to be dictated by a machine. While studies suggest that doctors are not very worried about AI replacing them in their jobs (Chen, Chen, et al, 2021) and are actually eager to adopt AI in their work (AMA, 2022), claims from computer scientists that AI will replace physicians has clearly led to some negative sentiment towards AI (Kim, 2018). A final factor that may dampen beliefs in the benevolence of AI is the ethics reified in the technology.…”

Section: Benevolencementioning

confidence: 99%

Augmenting physicians with artificial intelligence to transform healthcare: Challenges and opportunities

Agarwal,

Dugas,

Gao

2023

Economics Manag Strategy

View full text Add to dashboard Cite

We reflect on the progress and prospects of artificial intelligence (AI)‐powered transformation in healthcare from the perspective of front‐line clinical professionals responsible for care delivery. While there is considerable optimism about the potential of AI, critical gaps in understanding remain that represent fruitful opportunities for economics and management scholars. We outline the ways in which the strengths of AI can compensate for key limitations of physicians. We then focus on productive use of AI by physicians, highlighting the need for a deeper understanding of human‐AI teaming. We argue that productive teaming requires research on two critical issues: trust in AI and the redesign of clinical workflow to optimally accommodate artificial and human intelligence synergistically.

show abstract

“…With the growth of artificial intelligence (AI) applications in medicine, concern over fairness and transparency has also grown. A growing concern highlighted by multiple studies [1][2][3][4] is the phenomenon of algorithmic shortcutting, wherein DL models grasp superficial correlations in training data, potentially leading to biased or unreliable predictions. This concern holds particular weight in orthopedics, where machine learning is deployed for various applications, ranging from the detection of rare fractures to the classification of injuries and prediction of patient outcomes 5,6 .…”

Section: Introductionmentioning

confidence: 99%

Algorithmic shortcutting in medical image analysis

Koback,

Hill,

Barnum

et al. 2024

Medical Imaging 2024: Computer-Aided Diagnosis

View full text Add to dashboard Cite

As deep learning (DL) gains prominence in medical image analysis, its applications in orthopedics and broader medical contexts have expanded significantly. While machine learning algorithms show potential in using X-ray images for predicting outcomes, biases may arise due to insufficient preprocessing. One such source of bias is algorithmic shortcutting, a phenomenon wherein DL models inadvertently train on patterns within training data unrelated to the intended diagnostic content. Leveraging the Osteoarthritis Initiative (OAI) dataset and ResNet18, our convolutional neural network (CNN) could predict the clinical center from which X-rays originated with over 95% accuracy. Moreover, our investigation unveiled significant variations in patient demographics and outcomes between clinical centers, implying that discerning the originating clinical center could be a potential shortcut in predicting patient outcomes from X-rays. These findings underscore the importance of careful X-ray image preparation and analysis. More broadly, as deep learning models increasingly influence clinical decision-making, this work underscores the importance of addressing model shortcutting to enhance the transparency, interpretability, and efficacy of deep learning research across orthopedics and other medical domains.

show abstract

Algorithmic fairness in artificial intelligence for medicine and healthcare

Cited by 62 publications

References 274 publications

Fairness of Machine Learning Algorithms for Predicting Foregone Preventive Dental Care for Adults

Fairness of Machine Learning Algorithms for Predicting Foregone Preventive Dental Care for Adults

Augmenting physicians with artificial intelligence to transform healthcare: Challenges and opportunities

Algorithmic shortcutting in medical image analysis

Contact Info

Product

Resources

About