Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study

Abstract

BackgroundThe increasing adoption of electronic health records (EHRs) has been associated with a number of unintended negative consequences with provider efficiency and job satisfaction. To address this, there has been a dramatic increase in the use of medical scribes to perform many of the required EHR functions. Despite this rapid growth, little has been published on the training or assessment tools to appraise the safety and efficacy of scribe-related EHR activities. Given the number of reports documenting that other professional groups suffer from a number of performance errors in EHR interface and data gathering, scribes likely face similar challenges. This highlights the need for new assessment tools for medical scribes.ObjectiveThe objective of this study was to develop a virtual video-based simulation to demonstrate and quantify the variability and accuracy of scribes’ transcribed notes in the EHR.MethodsFrom a pool of 8 scribes in one department, a total of 5 female scribes, intent on pursuing careers in health care, with at least 6 months of experience were recruited for our simulation study. We created three simulated patient-provider scenarios. Each scenario contained a corresponding medical record in our simulation instance of our EHR. For each scenario, we video-recorded a standardized patient-provider encounter. Five scribes with at least 6 months of experience both with our EHR and in the specialty of the simulated cases were recruited. Each scribe watched the simulated encounter and transcribed notes into a simulated EHR environment. Transcribed notes were evaluated for interscribe variability and compared with a gold standard for accuracy.ResultsAll scribes completed all simulated cases. There was significant interscribe variability in note structure and content. Overall, only 26% of all data elements were unique to the scribe writing them. The term data element was used to define the individual pieces of data that scribes perceived from the simulation. Note length was determined by counting the number of words varied by 31%, 37%, and 57% between longest and shortest note between the three cases, and word economy ranged between 23% and 71%. Overall, there was a wide inter- and intrascribe variation in accuracy for each section of the notes with ranges from 50% to 76%, resulting in an overall positive predictive value for each note between 38% and 81%.ConclusionsWe created a high-fidelity, video-based EHR simulation, capable of assessing multiple performance indicators in medical scribes. In this cohort, we demonstrate significant variability both in terms of structure and accuracy in clinical documentation. This form of simulation can provide a valuable tool for future development of scribe curriculum and assessment of competency.

Promoting Quality Face-to-Face Communication during Ophthalmology Encounters in the Electronic Health Record Era

“…[24][25][26][27][28] However, some have suggested that using scribes results in lost nuance and cognitive processing time, 29 and relatively little is known about the safety and accuracy of scriberelated documentation. 30 Moreover, scribes may not always be financially feasible. Therefore, additional strategies to improve EHR use warrant further investigation.…”

Do You Know What Your Scribe Did Last Spring? The Impact of COVID-19 on Medical Scribe Workflow

“…5 Prior studies suggest deterioration with note veracity, which is potentially further confounded by these factors. 7 This concern is also exacerbated by the observation that the supervision of scribes is varied and unstructured. 12 This might be exaggerated by physical separation of scribes from their providers, reducing the opportunity for active collaboration and communication.…”

“…interscribe variability in the data elements recorded in their individual notes. 7,8 With the outbreak of the novel coronavirus disease 2019 (COVID-19), there has been a rapid transition to virtual care delivery, telecommuting, and redeploying personnel to meet evolving needs. 9 This raises the concern that when providers are still employing scribes, the scribes have been thrust into new clinical care delivery models with little training.…”

The Potential Impact of Scribes on Medical School Applicants and Medical Students with the New Clinical Documentation Guidelines

“…Two studies reviewing the quality of scribe documentation found significant variation in quality. 10,11 When supervisors spend time to improve scribe documentation, they are educating the scribes about appropriate use of medical language and the relative pertinence of information (e.g., choosing pertinent positives and negatives), which are also medical student milestones for developing clinical reasoning. This begs the question of whether this Btraining^time might be better spent with medical students.…”