Sentence Simplification with Memory-Augmented Neural Networks

Vu, Tu; Hu, Baotian; Munkhdalai, Tsendsuren; Yu, Hong

doi:10.18653/v1/n18-2013

Cited by 39 publications

(57 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As in the field of machine translation, early studies (Specia, 2010;Wubben et al, 2012;Xu et al, 2016) were mainly based on a statistical machine translation (Koehn et al, 2007;Post et al, 2013). Inspired by the success of neural machine translation (Bahdanau et al, 2015), recent studies (Nisioi et al, 2017;Zhang and Lapata, 2017;Vu et al, 2018;Guo et al, 2018;Zhao et al, 2018) use the encoder-decoder model with the attention mechanism. These studies do not consider the level of each sentence.…”

Section: Text Simplificationmentioning

confidence: 99%

Controllable Text Simplification with Lexical Constraint Loss

Nishihara

Kajiwara

Arase

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop

View full text Add to dashboard Cite

We propose a method to control the level of a sentence in a text simplification task. Text simplification is a monolingual translation task translating a complex sentence into a simpler and easier to understand the alternative. In this study, we use the grade level of the US education system as the level of the sentence. Our text simplification method succeeds in translating an input into a specific grade level by considering levels of both sentences and words. Sentence level is considered by adding the target grade level as input. By contrast, the word level is considered by adding weights to the training loss based on words that frequently appear in sentences of the desired grade level. Although existing models that consider only the sentence level may control the syntactic complexity, they tend to generate words beyond the target level. Our approach can control both the lexical and syntactic complexity and achieve an aggressive rewriting. Experiment results indicate that the proposed method improves the metrics of both BLEU and SARI.

show abstract

Section: Text Simplificationmentioning

confidence: 99%

Controllable Text Simplification with Lexical Constraint Loss

Nishihara

Kajiwara

Arase

2019

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop

View full text Add to dashboard Cite

show abstract

“…2 Nisioi et al (2017) was the first major application of Seq2Seq models to text simplification, applying a standard encoder-decoder approach with attention and beam search. Vu et al (2018) extended this framework to incorporate memory augmentation, which simultaneously performs lexical and syntactic simplification, allowing them to outperform standard Seq2Seq models.…”

Section: Related Workmentioning

confidence: 99%

Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification

Kriz¹,

Sedoc²,

Apidianaki³

et al. 2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Sentence simplification is the task of rewriting texts so they are easier to understand. Recent research has applied sequence-to-sequence (Seq2Seq) models to this task, focusing largely on training-time improvements via reinforcement learning and memory augmentation. One of the main problems with applying generic Seq2Seq models for simplification is that these models tend to copy directly from the original sentence, resulting in outputs that are relatively long and complex. We aim to alleviate this issue through the use of two main techniques. First, we incorporate content word complexities, as predicted with a leveled word complexity model, into our loss function during training. Second, we generate a large set of diverse candidate simplifications at test time, and rerank these to promote fluency, adequacy, and simplicity. Here, we measure simplicity through a novel sentence complexity model. These extensions allow our models to perform competitively with state-of-the-art systems while generating simpler sentences. We report standard automatic and human evaluation metrics. 1

show abstract

“…In the case of neural network methods, [43] enabled sentence simplification by deep reinforcement learning in which simplicity, relevance, and fluency are considered in the reward. [32] adapted a sequence-to-sequence model with a Neural Semantic Encoder [31] for the purpose of sentence simplification. [29] is a simple sequenceto-sequence model that can be efficiently used for in-domain and cross-domain sentence simplification.…”

Section: Related Work 21 Sentence Simplificationmentioning

confidence: 99%

“…The first approach is a method that generates a simple sentence from an input sentence by a statistical machine translation framework that interprets monolingual machine translation [22][36] [34] [35] [46]. The second approach is a method using a neural network [29] [32] [43]. In other NLP tasks (e.g., QA task, word embedding), the neural network method has had excellent results and is being actively researched.…”

Section: Introductionmentioning

confidence: 99%

Sentence Simplification from Non-Parallel Corpus with Adversarial Learning

Kawashima

Takagi

2019

IEEE/WIC/ACM International Conference on Web Intelligence

View full text Add to dashboard Cite

In this study, we propose sentence simplification from a non-parallel corpus with adversarial learning. In recent years, sentence simplification based on a statistical machine translation framework and neural networks have been actively studied. However, most methods require a large parallel corpus, which is expensive to build. In this paper, our purpose is sentence simplification with a nonparallel corpus in open data en-Wikipedia and Simple-Wikipedia articles. We use a style transfer framework with adversarial learning for learning by non-parallel corpus and adapted a prior work [by Barzilay et al.] to sentence simplification as a base framework. Furthermore, from the perspective of improving retention of sentence meaning, we add pretraining reconstruction loss and cycle consistency loss to the base framework. We also improve the sentence quality output from the proposed model as a result of the expansion.

show abstract

Sentence Simplification with Memory-Augmented Neural Networks

Cited by 39 publications

References 20 publications

Controllable Text Simplification with Lexical Constraint Loss

Controllable Text Simplification with Lexical Constraint Loss

Complexity-Weighted Loss and Diverse Reranking for Sentence Simplification

Sentence Simplification from Non-Parallel Corpus with Adversarial Learning

Contact Info

Product

Resources

About