Feasibility and Reliability Testing of Manual Electronic Health Record Reviews as a Tool for Timely Identification of Diagnostic Error in Patients at Risk

Abstract

Background Although diagnostic error (DE) is a significant problem, it remains challenging for clinicians to identify it reliably and to recognize its contribution to the clinical trajectory of their patients. The purpose of this work was to evaluate the reliability of real-time electronic health record (EHR) reviews using a search strategy for the identification of DE as a contributor to the rapid response team (RRT) activation. Objectives Early and accurate recognition of critical illness is of paramount importance. The objective of this study was to test the feasibility and reliability of prospective, real-time EHR reviews as a means of identification of DE. Methods We conducted this prospective observational study in June 2019 and included consecutive adult patients experiencing their first RRT activation. An EHR search strategy and a standard operating procedure were refined based on the literature and expert clinician inputs. Two physician-investigators independently reviewed eligible patient EHRs for the evidence of DE within 24 hours after RRT activation. In cases of disagreement, a secondary review of the EHR using a taxonomy approach was applied. The reviewers categorized patient experience of DE as Yes/No/Uncertain. Results We reviewed 112 patient records. DE was identified in 15% of cases by both reviewers. Kappa agreement with the initial review was 0.23 and with the secondary review 0.65. No evidence of DE was detected in 60% of patients. In 25% of cases, the reviewers could not determine whether DE was present or absent. Conclusion EHR review is of limited value in the real-time identification of DE in hospitalized patients. Alternative approaches are needed for research and quality improvement efforts in this field.

Defining Diagnostic Error: A Scoping Review to Assess the Impact of the National Academies’ Report Improving Diagnosis in Health Care

“…Three studies focused on measurement and surveillance of diagnostic errors. The prospective observational study by Soleimani et al, 31 focused on accuracy and timeliness, used medical record review that applied diagnostic criteria (e.g., a new diagnostic label within 24 hours after rapid response team and whether any features, indicative of that diagnosis, were present for greater than 6 hours before the first documentation of that new diagnosis). Where there was disagreement, a second review was conducted using the taxonomy delineating stages in the diagnostic process used in the study by Schiff et al 12 : history, physical exam, testing, assessment, referral/consultation, and follow-up.…”

“…Nine studies 20,23,24,26,29,30,32,33,35 indicated using the NASEM definition, 5 of those 20,23,24,30,32 operationalized it using a definition proposed before the NASEM report (see Table 3 for list of definitions). Three studies 21,31,34 operationalized error using existing definitions, and 4 studies 22,25,27,28 operationalized components of the NASEM definition (i.e., accuracy, timeliness, communication) for the purpose of the study and did not cite existing definitions. To capture content focus, we grouped studies according to the area of focus for which the definition was used: epidemiology, patient perspectives, measurement/ surveillance, and clinician perspectives.…”

Bedside Clinicians’ Perceptions on the Contributing Role of Diagnostic Errors in Acutely Ill Patient Presentation: A Survey of Academic and Community Practice

“…We did try to reach several members of the responding or admitting teams to gather multiple perceptions. Our acute hospital learning laboratory is developing strategies to identify DE in the EHR in near real-time, 22 and we plan to correlate the findings of DE from surveys with the EHR data.…”

“…Electronic health records (EHRs) cannot reliably capture all of the cues used by clinicians as they formulate a diagnosis. Discerning clinicians’ thought process is something that traditional measurement mechanisms including autopsies, 19,20 chart reviews, 21,22 and malpractice claims 23,24 often fail to do. When prompted, clinicians are well positioned to recognize errors and to lead efforts to mitigate their impact 13 .…”

Contributors to Diagnostic Error or Delay in the Acute Care Setting: A Survey of Clinical Stakeholders

“…3 As interest in this area has increased, studies have used various methods to better understand the prevalence and sources of DEOD. These methods include experimental simulations 4 ; reviewing charts, 5 patient and family medical complaints, 6 and malpractice claims 7 , 8 ; dashboards for proactive identification 9 ; physician surveys 10 - 12 or self-report 13 ; and a combined approach of observation and focus groups and interviews. 14 However, the precise rate of diagnostic error remains difficult to determine due to differences in methods used and how diagnostic error is defined.…”