Semantically Driven Sentence Fusion: Modeling and Evaluation

Abstract

Sentence fusion is the task of joining related sentences into coherent text. Current training and evaluation schemes for this task are based on single reference ground-truths and do not account for valid fusion variants. We show that this hinders models from robustly capturing the semantic relationship between input sentences. To alleviate this, we present an approach in which ground-truth solutions are automatically expanded into multiple references via curated equivalence classes of connective phrases. We apply this method to a large-scale dataset and use the augmented dataset for both model training and evaluation. To improve the learning of semantic representation using multiple references, we enrich the model with auxiliary discourse classification tasks under a multi-tasking framework. Our experiments highlight the improvements of our approach over state-of-the-art models. 1

EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Start

“…Following Geva et al (2019), we report Exact match, which is the percentage of exactly correctly predicted fusions. In addition to the T5 baseline and the text-editing baselines LASERTAG-GER , FELIX (Mallinson et al, 2020), and Seq2Edits, an autoregressive textediting model (Stahlberg and Kumar, 2020), we also report state-of-the-art sequence-to-sequence models ROBERTASHARE (Rothe et al, 2020b), based on ROBERTA large, and AugBERT (Ben-David et al, 2020), based on BERT base.…”

Comparative Snippet Generation

“…It is typically utilized as an intermediate step to improve downstream tasks. Ben-David et al (2020) train a model to learn both the discourse relation and discourse connective together in a multi-task framework. In our work, similar to Rothe et al (2020), we fuse two sentences together by training a model to learn the appropriate insertion of a discourse connective.…”