EASSE: Easier Automatic Sentence Simplification Evaluation

Alva-Manchego, Fernando; Martin, Louis; Scarton, Carolina; Specia, Lucia

doi:10.18653/v1/d19-3009

Cited by 63 publications

(54 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wubben et al (2012;Wang et al (2016). What is significant for TS are the results for automatic measures in Xu et al (2016) followed by Alva-Manchego et al (2019). We draw our inspiration from the authors, along with the results of Xu et al (2012) in our measure propositions.…”

Section: Style Transfer Methodsmentioning

confidence: 79%

An Empirical Study on Multi-Task Learning for Text Style Transfer and Paraphrase Generation

Bujnowski

Ryzhova²,

Choi

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics: Industry Track

View full text Add to dashboard Cite

The topic of this paper is neural multi-task training for text style transfer. We present an efficient method for neutral-to-style transformation using the transformer framework. We demonstrate how to prepare a robust model utilizing large paraphrases corpora together with a small parallel style transfer corpus. We study how much style transfer data is needed for a model on the example of two transformations: neutral-to-cute on internal corpus and modern-to-antique on publicly available Bible corpora. Additionally, we propose a synthetic measure for the automatic evaluation of style transfer models. We hope our research is a step towards replacing common but limited rule-based style transfer systems by more flexible machine learning models for both public and commercial usage.

show abstract

Section: Style Transfer Methodsmentioning

confidence: 79%

An Empirical Study on Multi-Task Learning for Text Style Transfer and Paraphrase Generation

Bujnowski

Ryzhova²,

Choi

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics: Industry Track

View full text Add to dashboard Cite

show abstract

“…Although some studies consider human judgments on grammaticality, meaning preservation and simplicity the most reliable method for evaluating the sentence simplification task, it is a common practice to use automatic metrics [Alva-Manchego et al 2019]. Following the WikiSplit work, we adopted the BLEU (Bilingual Evaluation Understudy) method [Papineni et al 2002] to validate the results.…”

Section: Resultsmentioning

confidence: 99%

Experimenting Sentence Split-and-Rephrase Using Part-of-Speech Labels

Neto¹,

Okano²,

Ruiz³

2020

Anais Do Symposium on Knowledge Discovery, Mining and Learning (KDMiLe 2020)

View full text Add to dashboard Cite

Text simplification (TS) is a natural language transformation process that reduces linguistic complexity while preserving semantics and retaining its original meaning. This work aims to present a research proposal for automatic simplification of texts, precisely a split-and-rephrase approach based on an encoder-decoder neural network model. The proposed method was trained against the WikiSplit English corpus with the help of a part-of-speech tagger and obtained a BLEU score validation of 74.72%. We also experimented with this trained model to split-and-rephrase sentences written in Portuguese with relative success, showing the method’s potential.

show abstract

“…In our experiments, we used the implementations of these metrics available in the EASSE package for automatic sentence simplification evaluation (Alva-Manchego et al, 2019). 5 We computed all the scores at sentence-level as in the experiment by Xu et al (2016), where they compared sentencelevel correlations of FKGL, BLEU and SARI with human ratings.…”

Section: Methodsmentioning

confidence: 99%

ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations

Alva-Manchego¹,

Martin²,

Bordes³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

In order to simplify a sentence, human editors perform multiple rewriting transformations: they split it into several shorter sentences, paraphrase words (i.e. replacing complex words or phrases by simpler synonyms), reorder components, and/or delete information deemed unnecessary. Despite these varied range of possible text alterations, current models for automatic sentence simplification are evaluated using datasets that are focused on a single transformation, such as lexical paraphrasing or splitting. This makes it impossible to understand the ability of simplification models in more realistic settings. To alleviate this limitation, this paper introduces ASSET, a new dataset for assessing sentence simplification in English. ASSET is a crowdsourced multi-reference corpus where each simplification was produced by executing several rewriting transformations. Through quantitative and qualitative experiments, we show that simplifications in ASSET are better at capturing characteristics of simplicity when compared to other standard evaluation datasets for the task. Furthermore, we motivate the need for developing better methods for automatic evaluation using ASSET, since we show that current popular metrics may not be suitable when multiple simplification transformations are performed.

show abstract

EASSE: Easier Automatic Sentence Simplification Evaluation

Cited by 63 publications

References 14 publications

An Empirical Study on Multi-Task Learning for Text Style Transfer and Paraphrase Generation

An Empirical Study on Multi-Task Learning for Text Style Transfer and Paraphrase Generation

Experimenting Sentence Split-and-Rephrase Using Part-of-Speech Labels

ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations

Contact Info

Product

Resources

About