SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization

Abstract

This paper introduces the SAMSum Corpus, a new dataset with abstractive dialogue summaries. We investigate the challenges it poses for automated summarization by testing several models and comparing their results with those obtained on a corpus of news articles. We show that model-generated summaries of dialogues achieve higher ROUGE scores than the model-generated summaries of news -in contrast with human evaluators' judgement. This suggests that a challenging task of abstractive dialogue summarization requires dedicated models and non-standard quality measures. To our knowledge, our study is the first attempt to introduce a high-quality chatdialogues corpus, manually annotated with abstractive summarizations, which can be used by the research community for further studies.

Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization

“…Global View and Discrete View In addition to the aforementioned two structured views, conversations can also be naturally viewed from a relatively coarse perspective, i.e., a global view that concatenates all utterances into one giant block (Gliwa et al, 2019), and a discrete view that separates each utterance into a distinct block Gliwa et al, 2019).…”

“…We evaluate our model on a large-scale dialogue summary dataset SAMSum (Gliwa et al, 2019) that has 14732 dialogues with human-written summaries. The data statistics are shown in Table 3.…”

“…There has been some recent research on conversation summarization such as directly deploying existing document summarization models (Gliwa et al, 2019) and exploring multi-sentence compression (Shang et al, 2018), however, most of them haven't utilized specific conversational structures, which refer to the way utterances are organized in order to make the conversation meaningful, enjoyable and understandable (Sacks et al, 1978), in dialogues -a key factor that differentiates dialogues from structured documents. As a way of using language socially of "doing things with words" together with other persons, the conversation has its own dynamic structures that organize utterances in certain orders to make the conversation meaningful, enjoyable, and understandable (Sacks et al, 1978).…”

“…As a way of using language socially of "doing things with words" together with other persons, the conversation has its own dynamic structures that organize utterances in certain orders to make the conversation meaningful, enjoyable, and understandable (Sacks et al, 1978). Although there are a few exceptions such as utilizing topic segmentation (Liu et al, 2019b;, dialogue acts (Goo and Chen, 2018) or key point sequence (Liu et al, 2019a) (Gliwa et al, 2019) with its topic view and stage view (extracted by our methods), and the human annotated summary. extensive expert annotations of discourse acts (Goo and Chen, 2018;Liu et al, 2019a), or only encode conversations based on their topics (Liu et al, 2019b), which fails to capture rich conversation structures in dialogues.…”

“…(2) We de-sign a multi-view sequence-to-sequence model that consists of a conversation encoder to encode different views and a multi-view decoder with multiview attention to generate dialogue summaries. (3) We perform experiments on a large-scale conversation summarization dataset, SAMSum (Gliwa et al, 2019), and demonstrate the effectiveness of our proposed methods. (4) We conduct thorough error analyses and discuss specific challenges that current approaches faced with this task.…”