FELIX: Flexible Text Editing Through Tagging and Insertion

Mallinson, Jonathan; Severyn, Aliaksei; Malmi, Eric; Garrido, Guillermo

doi:10.18653/v1/2020.findings-emnlp.111

Cited by 38 publications

(76 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pointer-generator style models (See et al, 2017;Xu et al, 2020) can accurately generate mostly extractive summaries by copying words from the source text via pointing. Text editing models Dong et al, 2019b;Mallinson et al, 2020) cast text generation as a sequence tagging problem with carefully selected edit operations required for the task. Others focus on improving content selection to better constrain the model to likely input phrases (Gehrmann et al, 2018) or by improving the representation of relevant input tokens .…”

Section: Related Workmentioning

confidence: 99%

Focus Attention: Promoting Faithfulness and Diversity in Summarization

Aralikatte¹,

Narayan²,

Maynez³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Professional summaries are written with document-level information, such as the theme of the document, in mind. This is in contrast with most seq2seq decoders which simultaneously learn to focus on salient content, while deciding what to generate, at each decoding step. With the motivation to narrow this gap, we introduce Focus Attention Mechanism, a simple yet effective method to encourage decoders to proactively generate tokens that are similar or topical to the input document. Further, we propose a Focus Sampling method to enable generation of diverse summaries, an area currently understudied in summarization. When evaluated on the BBC extreme summarization task, two state-of-the-art models augmented with Focus Attention generate summaries that are closer to the target and more faithful to their input documents, outperforming their vanilla counterparts on ROUGE and multiple faithfulness measures. We also empirically demonstrate that Focus Sampling is more effective in generating diverse and faithful summaries than top-k or nucleus samplingbased decoding methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Focus Attention: Promoting Faithfulness and Diversity in Summarization

Aralikatte¹,

Narayan²,

Maynez³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…To model the length, it is possible to use an autoregressive decoder or a separate model (Mansimov et al, 2019). Instead, we use an efficient non-autoregressive padded MLM approach by Mallinson et al (2020) which enables BERT to predict [PAD] symbols when infilling a fixed-length spans of n p [MASK] tokens.…”

Section: Padded Masked Language Modelsmentioning

confidence: 99%

“…Text-editing methods (Dong et al, 2019;Awasthi et al, 2019;Mallinson et al, 2020), that target monolingual sequence transduction tasks like sentence fusion, grammar correction, and text simplification, are typically more dataefficient than the traditional sequence-to-sequence methods, but they still require substantial amounts of parallel training examples to work well. When parallel source-target training pairs are difficult to obtain, it is often still possible to collect nonparallel examples for the source and the target domain separately.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Text Style Transfer with Padded Masked Language Models

Malmi¹,

Severyn²,

Rothe³

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

We propose MASKER, an unsupervised textediting method for style transfer. To tackle cases when no parallel source-target pairs are available, we train masked language models (MLMs) for both the source and the target domain. Then we find the text spans where the two models disagree the most in terms of likelihood. This allows us to identify the source tokens to delete to transform the source text to match the style of the target domain. The deleted tokens are replaced with the target MLM, and by using a padded MLM variant, we avoid having to predetermine the number of inserted tokens. Our experiments on sentence fusion and sentiment transfer demonstrate that MASKER performs competitively in a fully unsupervised setting. Moreover, in lowresource settings, it improves supervised methods' accuracy by over 10 percentage points when pre-training them on silver training data generated by MASKER.

show abstract

“…We hypothesize that BERT2BERT's strategy (Botha et al, 2018). 32.31 Dong et al (2019) 34.94 Xu et al (2016) 37.94 Mallinson et al (2020) 38.13 This work (no tags)…”

Section: Sentence Fusionmentioning

confidence: 99%

Seq2Edits: Sequence Transduction Using Span-level Edit Operations

Stahlberg¹,

Kumar²

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We propose Seq2Edits, an open-vocabulary approach to sequence editing for natural language processing (NLP) tasks with a high degree of overlap between input and output texts. In this approach, each sequence-to-sequence transduction is represented as a sequence of edit operations, where each operation either replaces an entire source span with target tokens or keeps it unchanged. We evaluate our method on five NLP tasks (text normalization, sentence fusion, sentence splitting & rephrasing, text simplification, and grammatical error correction) and report competitive results across the board. For grammatical error correction, our method speeds up inference by up to 5.2x compared to full sequence models because inference time depends on the number of edits rather than the number of target tokens. For text normalization, sentence fusion, and grammatical error correction, our approach improves explainability by associating each edit operation with a human-readable tag.

show abstract

FELIX: Flexible Text Editing Through Tagging and Insertion

Cited by 38 publications

References 35 publications

Focus Attention: Promoting Faithfulness and Diversity in Summarization

Focus Attention: Promoting Faithfulness and Diversity in Summarization

Unsupervised Text Style Transfer with Padded Masked Language Models

Seq2Edits: Sequence Transduction Using Span-level Edit Operations

Contact Info

Product

Resources

About