Artificial intelligence-assisted clinical decision support for childhood asthma management: A randomized clinical trial

Abstract

Rationale Clinical decision support (CDS) tools leveraging electronic health records (EHRs) have been an approach for addressing challenges in asthma care but remain under-studied through clinical trials. Objectives To assess the effectiveness and efficiency of Asthma-Guidance and Prediction System (A-GPS), an Artificial Intelligence (AI)-assisted CDS tool, in optimizing asthma management through a randomized clinical trial (RCT). Methods This was a single-center pragmatic RCT with a stratified randomization design conducted for one year in the primary care pediatric practice of the Mayo Clinic, MN. Children (<18 years) diagnosed with asthma receiving care at the study site were enrolled along with their 42 primary care providers. Study subjects were stratified into three strata (based on asthma severity, asthma care status, and asthma diagnosis) and were blinded to the assigned groups. Measurements Intervention was a quarterly A-GPS report to clinicians including relevant clinical information for asthma management from EHRs and machine learning-based prediction for risk of asthma exacerbation (AE). Primary endpoint was the occurrence of AE within 1 year and secondary outcomes included time required for clinicians to review EHRs for asthma management. Main results Out of 555 participants invited to the study, 184 consented for the study and were randomized (90 in intervention and 94 in control group). Median age of 184 participants was 8.5 years. While the proportion of children with AE in both groups decreased from the baseline (P = 0.042), there was no difference in AE frequency between the two groups (12% for the intervention group vs. 15% for the control group, Odds Ratio: 0.82; 95%CI 0.374–1.96; P = 0.626) during the study period. For the secondary end points, A-GPS intervention, however, significantly reduced time for reviewing EHRs for asthma management of each participant (median: 3.5 min, IQR: 2–5), compared to usual care without A-GPS (median: 11.3 min, IQR: 6.3–15); p<0.001). Mean health care costs with 95%CI of children during the trial (compared to before the trial) in the intervention group were lower than those in the control group (-$1,036 [-$2177, $44] for the intervention group vs. +$80 [-$841, $1000] for the control group), though there was no significant difference (p = 0.12). Among those who experienced the first AE during the study period (n = 25), those in the intervention group had timelier follow up by the clinical care team compared to those in the control group but no significant difference was found (HR = 1.93; 95% CI: 0.82–1.45, P = 0.10). There was no difference in the proportion of duration when patients had well-controlled asthma during the study period between the intervention and the control groups. Conclusions While A-GPS-based intervention showed similar reduction in AE events to usual care, it might reduce clinicians’ burden for EHRs review resulting in efficient asthma management. A larger RCT is needed for further studying the findings. Trial registration ClinicalTrials.gov Identifier: NCT02865967.

An individual-level socioeconomic measure for assessing algorithmic bias in health care settings: A case for HOUSES index

“…The study population and setting were described in our recent report. 30 Briefly, Olmsted County, is a virtually self-contained health care environment with only two health care systems provide clinical care to nearly all residents. About 98% of residents authorize use of their medical records for research.…”

“…34 Study Design and Subjects: The study design and subjects were described in our recent report. 30 Briefly, the study was designed as a cross-sectional study. We used the data from the training and testing cohorts that were used to develop ML algorithms applied to a previous study that developed a novel AI-assisted clinical decision support system named A-GPS (Asthma-Guidance and Prediction System), which is based on a single-center pragmatic randomized clinical trial (RCT).…”

“…A-GPS-based intervention in the original study provided clinicians with the summary of most relevant information for asthma management and the risk prediction for asthma exacerbation using ML algorithms. 30 It significantly reduced clinicians' burden for EHRs review resulting in more efficient asthma management by providing more actionable guidance. The focus of the original study was to assess the effectiveness of A-GPS on asthma outcomes (e.g., asthma exacerbation, asthma control, asthma-related health care utilization, asthma care quality and health care costs).…”

“…Details of the original study have been reported. 30 For an additional analysis, we used the subjects in the original study who met PAC definition but were not yet diagnosed with asthma at the time of enrollment. This study (IRB number:15-004435) was approved by the Mayo Clinic Institutional Review Board (IRB).…”

“…status59 variables were extracted from EHR in the 3 years prior to the study index date. Additionally, we assessed data validity defined as having ICD-9 codes for asthma among those who met PAC definition but were not yet diagnosed with asthma at the time of the study 30. Specifically, we previously reported a significant number of children had undiagnosed asthma by comparing asthma prevalence by ICD code-based asthma ascertainment with that by NLP-based ascertainment using PAC (sensitivity 31% (ICD-9) vs. 81% (NLP logic) and 85% (NLP ML)).60- 62 Subject characteristics used in the study In this presented work, we quantified algorithmic bias for two ML algorithms (NB and GBM models for predicting 1-year AE risk among pediatric asthmatics) by demographic factors (age, sex, race/ethnicity), SES (HOUSES), and chronic condition.…”