Generating Medical Reports from Patient-Doctor Conversations Using Sequence-to-Sequence Models

Abstract

We discuss automatic creation of medical reports from ASR-generated patient-doctor conversational transcripts using an end-to-end neural summarization approach.We explore both recurrent neural network (RNN) and Transformer-based sequence-to-sequence architectures for summarizing medical conversations. We have incorporated enhancements to these architectures, such as the pointer-generator network that facilitates copying parts of the conversations to the reports, and a hierarchical RNN encoder that makes RNN training three times faster with long inputs. A comparison of the relative improvements from the different model architectures over an oracle extractive baseline is provided on a dataset of 800k orthopedic encounters. Consistent with observations in literature for machine translation and related tasks, we find the Transformer models outperform RNN in accuracy, while taking less than half the time to train. Significantly large wins over a strong oracle baseline indicate that sequenceto-sequence modeling is a promising approach for automatic generation of medical reports, in the presence of data at scale.

The digital scribe in clinical practice: a scoping review and research agenda

“…After screening the titles and abstracts of these articles, we assessed 144 full-text articles for eligibility. We included 20 articles [19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38] for our analysis (Fig. 1 and Supplementary Table 2).…”

Hierarchical Speaker-Aware Sequence-to-Sequence Model for Dialogue Summarization

“…Summaries predicted by our model are evaluated with ROUGE scores. Besides the baselines mention above, the base pre-trained model BART-base and transformerPGN [19] are also evaluated on the SAMsum dataset. Then we performed main experiments and an ablation experiment on our model.…”

Towards Automating Medical Scribing : Clinic Visit Dialogue2Note Sentence Alignment and Snippet Summarization

“…They experimented with different LSTM sequence-tosequence methods, various attention mechanisms, pointer generator mechanisms, and topic information additions. (Enarvi et al, 2020) performed similar work with sequence-to-sequence methods on a corpus of 800K orthopaedic ASR generated transcripts and notes; (Krishna et al, 2020) on a corpus of 6862 visits of transcripts annotated with clinical note summary sentences. Unlike most of previous works, our task generates clinical note sentences from labeled transcript snippets, which are at times overlapping and discontinuous.…”