Parallel Iterative Edit Models for Local Sequence Transduction

Awasthi, Abhijeet; Sarawagi, Sunita; Goyal, Rasna; Ghosh, Sabyasachi; Piratla, Vihari

doi:10.48550/arxiv.1910.02893

Cited by 6 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As I'm new here, I'm lost and don't know where my hotel is. One branch of the work is language-dependent approaches, like PIE (Awasthi et al, 2019) and GECToR (Omelianchuk et al, 2020). They predict a sequence of token-level edit operations including a number of manually designed languagespecific operations like changing verb forms (e.g., VBZ→VBD) and prepositions (e.g., in→on).…”

Section: Related Workmentioning

confidence: 99%

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction

Chen¹,

Ge²,

Zhang³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We propose a novel language-independent approach to improve the efficiency for Grammatical Error Correction (GEC) by dividing the task into two subtasks: Erroneous Span Detection (ESD) and Erroneous Span Correction (ESC). ESD identifies grammatically incorrect text spans with an efficient sequence tagging model. Then, ESC leverages a seq2seq model to take the sentence with annotated erroneous spans as input and only outputs the corrected text for these spans. Experiments show our approach performs comparably to conventional seq2seq approaches in both English and Chinese GEC benchmarks with less than 50% time cost for inference.

show abstract

Section: Related Workmentioning

confidence: 99%

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction

Chen¹,

Ge²,

Zhang³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…Grammatical Error Correction (GEC), which aims to detect and correct grammatical errors of text, is an important and active research area in natural language processing (Ng et al, 2014;Napoles et al, 2017;Ge et al, 2018;Awasthi et al, 2019;Omelianchuk et al, 2020;Rothe et al, 2021). In this work, we study Chinese GEC (Chang, 1995;Huang et al, 2000;Yu and Li, 2014;Zhang et al, 2015;, which has unique challenges as Chinese language does not have word delimiters (i.e., spaces) in written sentences.…”

Section: Introductionmentioning

confidence: 99%

Pretraining Chinese BERT for Detecting Word Insertion and Deletion Errors

Zhang¹,

Dai²,

Tang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Chinese BERT models achieve remarkable progress in dealing with grammatical errors of word substitution. However, they fail to handle word insertion and deletion because BERT assumes the existence of a word at each position. To address this, we present a simple and effective Chinese pretrained model. The basic idea is to enable the model to determine whether a word exists at a particular position. We achieve this by introducing a special token [null], the prediction of which stands for the non-existence of a word. In the training stage, we design pretraining tasks such that the model learns to predict [null] and real words jointly given the surrounding context. In the inference stage, the model readily detects whether a word should be inserted or deleted with the standard masked language modeling function. We further create an evaluation dataset to foster research on word insertion and deletion. It includes human-annotated corrections for 7,726 erroneous sentences. Results show that existing Chinese BERT performs poorly on detecting insertion and deletion errors. Our approach significantly improves the F1 scores from 24.1% to 78.1% for word insertion and from 26.5% to 68.5% for word deletion, respectively.

show abstract

“…Consequentially, the seq2seq models may suffer from these potential risks in practice which conflict with the second primary goal of the GEC task. Recently, sequence tagging methods (Malmi et al 2019;Awasthi et al 2019;Omelianchuk et al 2020;Stahlberg and Kumar 2020) consider GEC as a text editing task by detecting and applying edits to the original sentences, therefore bypassing the above mentioned problems of seq2seq models. Nevertheless, the edits are usually constrained by human designed or automatically generated lexical rules (Omelianchuk et al 2020;Stahlberg and Kumar 2020) and vocabularies (Awasthi et al 2019;Malmi et al 2019), which limits the generality and transferability of these methods.…”

Section: Introductionmentioning

confidence: 99%

“…Recently, sequence tagging methods (Malmi et al 2019;Awasthi et al 2019;Omelianchuk et al 2020;Stahlberg and Kumar 2020) consider GEC as a text editing task by detecting and applying edits to the original sentences, therefore bypassing the above mentioned problems of seq2seq models. Nevertheless, the edits are usually constrained by human designed or automatically generated lexical rules (Omelianchuk et al 2020;Stahlberg and Kumar 2020) and vocabularies (Awasthi et al 2019;Malmi et al 2019), which limits the generality and transferability of these methods. Moreover, when it comes to corrections which need longer insertions, most of these sequence tagging methods rely on iterative corrections, which can reduce the fluency.…”

Section: Introductionmentioning

confidence: 99%

Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation

Guo

Zhang

et al. 2022

AAAI

View full text Add to dashboard Cite

The task of Grammatical Error Correction (GEC) has received remarkable attention with wide applications in Natural Language Processing (NLP) in recent years. While one of the key principles of GEC is to keep the correct parts unchanged and avoid over-correction, previous sequence-to-sequence (seq2seq) models generate results from scratch, which are not guaranteed to follow the original sentence structure and may suffer from the over-correction problem. In the meantime, the recently proposed sequence tagging models can overcome the over-correction problem by only generating edit operations, but are conditioned on human designed language-specific tagging labels. In this paper, we combine the pros and alleviate the cons of both models by proposing a novel Sequence-to-Action (S2A) module. The S2A module jointly takes the source and target sentences as input, and is able to automatically generate a token-level action sequence before predicting each token, where each action is generated from three choices named SKIP, COPY and GENerate. Then the actions are fused with the basic seq2seq framework to provide final predictions. We conduct experiments on the benchmark datasets of both English and Chinese GEC tasks. Our model consistently outperforms the seq2seq baselines, while being able to significantly alleviate the over-correction problem as well as holding better generality and diversity in the generation results compared to the sequence tagging models.

show abstract

Parallel Iterative Edit Models for Local Sequence Transduction

Cited by 6 publications

References 0 publications

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction

Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction

Pretraining Chinese BERT for Detecting Word Insertion and Deletion Errors

Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation

Contact Info

Product

Resources

About