Controllable Text Simplification with Lexical Constraint Loss

Nishihara, Daiki; Kajiwara, Tomoyuki; Arase, Yasuji

doi:10.18653/v1/p19-2036

Cited by 48 publications

(63 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The goal of text simplification is to transform text into a variant that is more broadly accessible to a wide variety of readers while preserving the content. While this has been accomplished using a range of approaches (Shardlow, 2014), most text simplification research has focused on fully-automated approaches (Xu et al, 2016;Zhang and Lapata, 2017;Nishihara et al, 2019). However, in some domains, such as healthcare, using fully-automated text simplification is not appropriate since it is critical that the important information is preserved fully during the simplification process.…”

Section: Introductionmentioning

confidence: 99%

AutoMeTS: The Autocomplete for Medical Text Simplification

Van¹,

Kauchak²,

Leroy³

2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

The goal of text simplification (TS) is to transform difficult text into a version that is easier to understand and more broadly accessible to a wide variety of readers. In some domains, such as healthcare, fully automated approaches cannot be used since information must be accurately preserved. Instead, semi-automated approaches can be used that assist a human writer in simplifying text faster and at a higher quality. In this paper, we examine the application of autocomplete to text simplification in the medical domain. We introduce a new parallel medical data set consisting of aligned English Wikipedia with Simple English Wikipedia sentences and examine the application of pretrained neural language models (PNLMs) on this dataset. We compare four PNLMs (BERT, RoBERTa, XLNet, and GPT-2), and show how the additional context of the sentence to be simplified can be incorporated to achieve better results (6.17% absolute improvement over the best individual model). We also introduce an ensemble model that combines the four PNLMs and outperforms the best individual model by 2.1%, resulting in an overall word prediction accuracy of 64.52%.

show abstract

Section: Introductionmentioning

confidence: 99%

AutoMeTS: The Autocomplete for Medical Text Simplification

Van¹,

Kauchak²,

Leroy³

2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Recently, Kriz et al (2019) used two techniques to solve the problem that the model tends to copy words directly, resulting in a long and complicated output sentence. Nishihara et al (2019) proposed a method to simplify the original sentences to different level sentences. Different from most sequenceto-sequence models, Dong et al (2019) proposed a neural programmer-interpreter approach to predict explicit edit operations directly.…”

Section: Related Workmentioning

confidence: 99%

On the Helpfulness of Document Context to Sentence Simplification

Sun¹,

Lin

Wan

2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Most of the research on text simplification is limited to sentence level nowadays. In this paper, we are the first to investigate the helpfulness of document context on sentence simplification and apply it to the sequence-to-sequence model. We firstly construct a sentence simplification dataset in which the contexts for the original sentence are provided by Wikipedia corpus. The new dataset contains approximately 116K sentence pairs with context. We then propose a new model that makes full use of the context information. Our model uses neural networks to learn the different effects of the preceding sentences and the following sentences on the current sentence and applies them to the improved transformer model. Evaluated on the newly constructed dataset, our model achieves 36.52 on SARI value, which outperforms the best performing model in the baselines by 2.46 (7.22%), indicating that context indeed helps improve sentence simplification. In the ablation experiment, we show that using either the preceding sentences or the following sentences as context can significantly improve simplification.

show abstract

“…Kriz et al [36] modified Seq2seq model by improving training loss function and decoding method of inference time. Nishihara et al [37] proposed controllable text simplification model which improved Seq2seq model with lexical constraint loss, but Nishihara's method depended heavily on dataset because it need the target sentence with sentence level label. These models had achieved a certain degree of good results in sentence simplification task.…”

Section: Related Workmentioning

confidence: 99%

Sentence Simplification Based on Multi-Stage Encoder Model

Zhang

Deng

2019

IEEE Access

View full text Add to dashboard Cite

Sentence simplification aims to simplify a complex sentences while retaining its main idea. It is one of the most important tasks in natural language processing. Recent works addressed the task with sequence-to-sequence(Seq2seq) model. However, these conventional Seq2seq models usually based on a single-stage encoder, which only read the source complex sentence once, as a result, it was hard to extract the representational features of the source sentence precisely. To resolve the problem, we proposed a multi-stage encoder based Seq2seq model for sentence simplification. Specificly, there were three stages in the encoder of proposed model, namely N-gram reading stage, glance-over stage and final encoding stage. The N-gram reading stage catched N-gram feature embedding for other stage and the glance-over stage extracted local and global information about the source sentence. The final encoding stage took advantage of the information extracted by the former two stage to encode source sentence better. Then, it introduced a novel attention connection method which could help the decoder to make full use of the information of encoder. Experiments on three public datasets demonstrated the proposed model that outperforms state-ofthe-art baseline simplification systems.

show abstract

Controllable Text Simplification with Lexical Constraint Loss

Cited by 48 publications

References 14 publications

AutoMeTS: The Autocomplete for Medical Text Simplification

AutoMeTS: The Autocomplete for Medical Text Simplification

On the Helpfulness of Document Context to Sentence Simplification

Sentence Simplification Based on Multi-Stage Encoder Model

Contact Info

Product

Resources

About