Automatic Rephrasing of Transcripts-based Action Items

Abstract

The automated transcription of spoken language, and meetings, in particular, is becoming more widespread as automatic speech recognition systems are becoming more accurate. This trend has significantly accelerated since the outbreak of the COVID-19 pandemic, which led to a major increase in the number of online meetings. However, the transcription of spoken language has not received much attention from the NLP community compared to documents and other forms of written language. In this paper, we study a variation of the summarization problem over the transcription of spoken language: given a transcribed meeting, and an action item (i.e., a commitment or request to perform a task), our goal is to generate a coherent and self-contained rephrasing of the action item. To this end, we compiled a novel dataset of annotated meeting transcripts, including human rephrasing of action items. We use state-of-the-art supervised text generation techniques and establish a strong baseline based on BART and UniLM (two pretrained transformer models). Due to the nature of natural speech, language is often broken and incomplete and the task is shown to be harder than an analogous task over email data. Particularly, we show that the baseline models can be greatly improved once models are provided with additional information. We compare two approaches: one incorporating features extracted by coreferenceresolution. Additional annotations are used to train an auxiliary model to detect the relevant context in the text. Based on the systematic human evaluation, our best models exhibit nearhuman-level rephrasing capability on a constrained subset of the problem.

Development and Evaluation of a Digital Scribe: Conversation Summarization Pipeline for Emergency Department Counseling Sessions towards Reducing Documentation Burden

“…The differential performance across various information categories illuminates the necessity for enhancing model recognition capabilities. [46] As the accuracy rates across most tags are promising, they also highlight the disparity in the model’s ability to uniformly identify and convey the full spectrum of clinically relevant information present in the reference summaries. [47] In a zero-shot context, each model performed relatively worse than their fine-tuned counterparts.…”

Evaluation of a Digital Scribe: Conversation Summarization for Emergency Department Consultation Calls

“…The differential performance across various information categories illuminates the necessity for enhancing model recognition capabilities. 57 As the accuracy rates across most tags are promising, they also highlight the disparity in the model's ability to uniformly identify and convey the full spectrum of clinically relevant information present in the reference summaries. 58 In a zero-shot context, each model performed relatively worse than their fine-tuned counterparts.…”

Action-Item-Driven Summarization of Long Meeting Transcripts

“…Although recording action items is an important part of many meeting summaries, the issue has been ignored in prior work. This problem was first introduced by Cohen et al [2], but little progress has been made since. To solve this, we use a public dataset 2 from a GitHub repository that contains 2750 dialogue statements as well as corresponding labels for whether a statement contains action items or not.…”