Controllable Text Simplification with Explicit Paraphrasing

Abstract

Text Simplification improves the readability of sentences through several rewriting transformations, such as lexical paraphrasing, deletion, and splitting. Current simplification systems are predominantly sequence-to-sequence models that are trained end-to-end to perform all these operations simultaneously. However, such systems limit themselves to mostly deleting words and cannot easily adapt to the requirements of different target audiences. In this paper, we propose a novel hybrid approach that leverages linguistically-motivated rules for splitting and deletion, and couples them with a neural paraphrasing model to produce varied rewriting styles. We introduce a new data augmentation method to improve the paraphrasing capability of our model. Through automatic and manual evaluations, we show that our proposed model establishes a new state-ofthe art for the task, paraphrasing more often than the existing systems, and can control the degree of each simplification operation applied to the input texts. 1

GRS: Combining Generation and Revision in Unsupervised Sentence Simplification

“…Building on a Seq2Seq model, Zhang and Lapata (2017) used reinforcement learning to optimize a reward based on simplicity, fluency and relevance. Recent methods build on transformer (Vaswani et al, 2017) models, by integrating external databases containing simplification rules (Zhao et al, 2018), using an additional loss function to generate diverse outputs (Kriz et al, 2019), combining syntactic rules (Maddela et al, 2021), and conditioning on length and syntactic and lexical complexity features (Martin et al, 2020a).…”

“…Simplification methods can also be categorized as supervised or unsupervised. Supervised methods tend to have better performance, but require aligned complex-simple sentence pairs for training (Zhang and Lapata, 2017;Guo et al, 2018;Kriz et al, 2019;Martin et al, 2020a,b;Maddela et al, 2021). Unsupervised methods do not need such training data but do not perform as well (Surya et al, 2019;Kumar et al, 2020;Zhao et al, 2020).…”

Evaluating Factuality in Text Simplification

“…We then evaluate outputs from several modern simplification models (Zhang and Lapata, 2017;Dong et al, 2019;Martin et al, 2020;Maddela et al, 2021), as well as a fine-tuned T5 (Raffel et al, 2020) model. Compared to RNN-based models, Transformer-based ones tend to have less severe deletion and substitution errors; however, the pre-trained T5 produced more hallucinations on the more abstractive Newsela dataset.…”

Neural semi-Markov CRF for Monolingual Word Alignment

“…It is useful for improving the interpretability in natural language understanding tasks, including semantic textual similarity (Li and Srikumar, 2016) and question answering (Yao, 2014). Monolingual word alignment can also support the analysis of human editing operations (Figure 1) and improve model performance for text-to-text generation tasks, such as text simplification (Maddela et al, 2021) and neutralizing biased language (Pryzant et al, 2020). It has also been shown to be helpful for data augmentation and label projection With Canadian collaborators, Lloyd went on to conduct laboratory simulations of his model.…”

“…More specifically, we sample from the exact test set used in Table2inMaddela et al (2021).6 This annotator has annotated MultiMWA-MTRef. 7 https://arxiv.org/ 8 https://github.com/pkubowicz/ opendetex…”