Bias and fairness assessment of a natural language processing opioid misuse classifier: detection and mitigation of electronic health record data disadvantages across racial subgroups

Abstract

Objectives To assess fairness and bias of a previously validated machine learning opioid misuse classifier. Materials & Methods Two experiments were conducted with the classifier’s original (n = 1000) and external validation (n = 53 974) datasets from 2 health systems. Bias was assessed via testing for differences in type II error rates across racial/ethnic subgroups (Black, Hispanic/Latinx, White, Other) using bootstrapped 95% confidence intervals. A local surrogate model was estimated to interpret the classifier’s predictions by race and averaged globally from the datasets. Subgroup analyses and post-hoc recalibrations were conducted to attempt to mitigate biased metrics. Results We identified bias in the false negative rate (FNR = 0.32) of the Black subgroup compared to the FNR (0.17) of the White subgroup. Top features included “heroin” and “substance abuse” across subgroups. Post-hoc recalibrations eliminated bias in FNR with minimal changes in other subgroup error metrics. The Black FNR subgroup had higher risk scores for readmission and mortality than the White FNR subgroup, and a higher mortality risk score than the Black true positive subgroup (P < .05). Discussion The Black FNR subgroup had the greatest severity of disease and risk for poor outcomes. Similar features were present between subgroups for predicting opioid misuse, but inequities were present. Post-hoc mitigation techniques mitigated bias in type II error rate without creating substantial type I error rates. From model design through deployment, bias and data disadvantages should be systematically addressed. Conclusion Standardized, transparent bias assessments are needed to improve trustworthiness in clinical machine learning models.

Evaluation and Mitigation of Racial Bias in Clinical Machine Learning Models: Scoping Review

“…Applications of ML were varied and involved diagnosis, outcome prediction, and clinical score prediction performed on data sets including images, diagnostic studies, clinical text, and clinical variables. Furthermore, 1 (8%) study described a model in routine clinical use [ 36 ], 2 (17%) examined prospectively validated clinical models [ 35 , 39 ], and the remaining 9 (75%) described internally validated models.…”

“…Of the 12 studies, 5 (42%) published code used for analysis, 3 (25%) made model development code available [ 34 , 36 , 39 ], 2 (17%) published bias analysis code [ 33 , 36 ], 1 (8%) published code relevant to debiasing [ 30 ], and 1 (8%) published data selection code [ 33 ]. In addition, 1 (8%) study used publicly available code for analysis [ 31 ], and code was specified as available upon request in 1 (8%) study [ 35 ].…”

“…In addition, 1 (8%) study used publicly available code for analysis [ 31 ], and code was specified as available upon request in 1 (8%) study [ 35 ]. Bias of an ML model was evaluated using an external database in 8 (67%) studies [ 30 - 34 , 37 , 38 ], single-institutional data in 3 (25%) studies [ 35 , 36 , 40 ], and data from 2 institutions in 2 (17%) studies [ 29 , 39 ]. No institutional data sets were published.…”

Development and multimodal validation of a substance misuse algorithm for referral to treatment using artificial intelligence (SMART-AI): a retrospective deep learning study

“…Post-hoc mitigation methods in the opioid misuse classifier were previously shown to improve disparities and could be considered before deployment. 28 Close observation is still required during deployment to identify unintended consequences. Implicit bias in provider notes remains a real problem in treatment of patients, especially in patients with substance misuse.…”

Machine Learning Approaches for Electronic Health Records Phenotyping: A Methodical Review

“…It is becoming increasingly apparent that ML models are prone to bias that can harm marginalized groups [159,160]. Only 1 article evaluated algorithmic fairness [161]. Fairness must be integrated into phenotyping in the future.…”