Monitoring Diagnostic Safety Risks in Emergency Departments: Protocol for a Machine Learning Study

Abstract

Background Diagnostic decision making, especially in emergency departments, is a highly complex cognitive process that involves uncertainty and susceptibility to errors. A combination of factors, including patient factors (eg, history, behaviors, complexity, and comorbidity), provider-care team factors (eg, cognitive load and information gathering and synthesis), and system factors (eg, health information technology, crowding, shift-based work, and interruptions) may contribute to diagnostic errors. Using electronic triggers to identify records of patients with certain patterns of care, such as escalation of care, has been useful to screen for diagnostic errors. Once errors are identified, sophisticated data analytics and machine learning techniques can be applied to existing electronic health record (EHR) data sets to shed light on potential risk factors influencing diagnostic decision making. Objective This study aims to identify variables associated with diagnostic errors in emergency departments using large-scale EHR data and machine learning techniques. Methods This study plans to use trigger algorithms within EHR data repositories to generate a large data set of records that are labeled trigger-positive or trigger-negative, depending on whether they meet certain criteria. Samples from both data sets will be validated using medical record reviews, upon which we expect to find a higher number of diagnostic safety events in the trigger-positive subset. Machine learning will be used to evaluate relationships between certain patient factors, provider-care team factors, and system-level risk factors and diagnostic safety signals in the statistically matched groups of trigger-positive and trigger-negative charts. Results This federally funded study was approved by the institutional review board of 2 academic medical centers with affiliated community hospitals. Trigger queries are being developed at both organizations, and sample cohorts will be labeled using the triggers. Machine learning techniques such as association rule mining, chi-square automated interaction detection, and classification and regression trees will be used to discover important variables that could be incorporated within future clinical decision support systems to help identify and reduce risks that contribute to diagnostic errors. Conclusions The use of large EHR data sets and machine learning to investigate risk factors (related to the patient, provider-care team, and system-level) in the diagnostic process may help create future mechanisms for monitoring diagnostic safety. International Registered Report Identifier (IRRID) DERR1-10.2196/24642

Diagnostic error rates and associated factors for lower gastrointestinal perforation

“…In Japan, the availability of urgent radiology reports is low, with only 26.9% cases reported in this study. Communication barriers between physicians and radiologists are due to a variety of factors, such as system factors (e.g., health information technology, crowding, shift-based work, and interruptions) 17 , 18 . Our results suggested that the unavailability of urgent radiology reports may be related to delayed diagnosis.…”

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach

“…The DSC was designed with the intent of identifying patients at risk for DE ( Table 2 ) based on patient, clinical, and system factors identified from the literature 5 , 45–48 and preliminary analysis of a DE case cohort at our institution. 6 , 25 , 29 By identifying at-risk patients, the DSC could then suggest actions to clinicians (such as recommending a DTO ) to mitigate DEs from occurring in these specific patients.…”

“… Risk factors identified from the literature 5 , 45–48 and our case cohort analysis 6 , 25 , 29 include: Altered mental status, Delirium, Dementia, Depression, Bipolar, Psychosis, or End-Stage Renal Disease on EHR Problem List Primary language not English 3 or more subspecialty consultants Inter-hospital transfer 2 or more outpatient visits within 10 d prior to admission A prior hospitalization within 7 d of index hospitalization Emergency department visit for undifferentiated sign or symptom within 10 d of admission 1 daytime responding clinician and 1 attending change, or 3 or more different attendings in last 72 h New or increasing oxygen requirement 2 or more blood gases (arterial or venous) resulted within 24 h High risk for clinical deterioration based on Epic’s deterioration index …”

“…7 , 25 , 29 Risk factors reported in the literature and common cognitive biases encountered by clinicians were included. 4 , 5 , 45 , 47 , 50 , 51 Finally, we presented an initial prototype of the DTO to clinical stakeholders and subject matter experts and incorporated their input. We tested the working prototype with clinicians (MDs, APPs, and RNs) and solicited feedback during testing sessions.…”