An automated medical scribe for documenting clinical encounters

Abstract

A medical scribe is a clinical professional who charts patient-physician encounters in real time, relieving physicians of most of their administrative burden and substantially increasing productivity and job satisfaction. We present a complete implementation of an automated medical scribe. Our system can serve either as a scalable, standardized, and economical alternative to human scribes; or as an assistive tool for them, providing a first draft of a report along with a convenient means to modify it. This solution is, to our knowledge, the first automated scribe ever presented and relies upon multiple speech and language technologies, including speaker diarization, medical speech recognition, knowledge extraction, and natural language generation.

Challenges of developing a digital scribe to reduce clinical documentation burden

“…17 Having multiple speakers participating in the conversation and differentiating them in the audio (speaker diarization) also adds a level of complexity and potential errors to ASR. 7 Recent work has shown the use of a recurrent neural network transducer significantly lowered diarization errors for audio recordings of clinical conversations between physicians and patients. 20 Even with ideal recording equipment, ASR of conversational speech is more vulnerable to errors.…”

“…22,23 Medical conversations have different statistical properties than medical dictations, meaning that ASR trained with dictations is likely to underperform with medical conversations. 7 After conversion from speech to text, NLP techniques that perform well on grammatically correct sentences break down with conversational speech because of the lack of punctuation and sentence boundaries, grammatical differences between spoken and written language, and lack of structure. 7,24,25 CHALLENGE 2: STRUCTURING CLINICIAN-PATIENT CONVERSATIONS ASR produces a transcript of the clinician-patient conversation that lacks clear boundaries and structure due to the unconstrained nature of conversations.…”

“…A digital scribe is an automated clinical documentation system able to capture the clinician-patient conversation and then generate the documentation for the encounter, like the function performed by human medical scribes. [5][6][7][8][9] In theory, a digital scribe would enable a clinician to fully engage with a patient, maintain eye contact, and eliminate the need to split attention by turning to a computer to manually document the encounter. Reducing the time and effort invested by clinicians in the documentation process also has the potential to increase productivity, decrease clinician burnout, and improve the clinician-patient relationship, leading to higher quality and patient-centered care.…”

“…Digital scribes can also be referred to as autoscribes, automated scribes, virtual medical scribes, artificial intelligence (AI) powered medical notes, speech recognition-assisted documentation, and smart medical assistants. 6,7 To generate medical notes for the clinician-patient encounter, a digital scribe must be able to: (1) record the clinician-patient conversation, (2) convert the audio to text, and (3) extract salient information from the text and summarize the information (Fig. 1).…”

“…The implementation of a digital scribe consists of a pipeline of speech-processing and natural language processing (NLP) modules. 7 Recently, advances in AI, machine learning (ML), NLP, natural language understanding, and automatic speech recognition (ASR), have raised the prospect of deploying effective and reliable digital scribes in clinical practice.…”