The CoNLL-2014 Shared Task on Grammatical Error Correction

Abstract

The CoNLL-2014 shared task was devoted to grammatical error correction of all error types. In this paper, we give the task definition, present the data sets, and describe the evaluation metric and scorer used in the shared task. We also give an overview of the various approaches adopted by the participating teams, and present the evaluation results. Compared to the CoNLL-2013 shared task, we have introduced the following changes in CoNLL-2014: (1) A participating system is expected to detect and correct grammatical errors of all types, instead of just the five error types in CoNLL-2013; (2) The evaluation metric was changed from F 1 to F 0.5 , to emphasize precision over recall; and (3) We have two human annotators who independently annotated the test essays, compared to just one human annotator in CoNLL-2013.

The Columbia System in the QALB-2014 Shared Task on Arabic Error Correction

“…The two most recent CoNLL shared tasks were devoted to grammatical error correction for non-native writers (Ng et al, 2013;Ng et al, 2014).…”

GWU-HASP: Hybrid Arabic Spelling and Punctuation Corrector

“…It is worth noting that by comparison, punctuation errors only constituted 4% of the English data in CoNLL 2013 Shared Task on English Grammatical Error Correction (Ng et al, 2013) and were not evaluated or handled by any participant. In HASP, we focus on 6 punctuation marks: comma, colon, semi-colon, exclamation mark, question mark and period.…”

Correcting Grammatical Verb Errors

“…7 We observe the relatively low recall obtained by the models. Error correction models tend to have low recall (see, for example, the recent shared tasks on ESL error correction (Dale and Kilgarriff, 2011;Dale et al, 2012;Ng et al, 2013)). The key reason for the low recall is the error sparsity: over 95% of verbs are correct, as shown in Table 9.…”

“…We also evaluate several methods for selecting verb candidates and show the significance of this step for improving verb error correction performance, while earlier studies do not discuss this aspect of the problem. In the CoNLL shared task (Ng et al, 2013) that included verb errors in agreement and form, the participating teams did not provide details on how specific challenges were handled, but the University of Illinois system obtained the highest score on the verb sub-task, even though all teams used similar resources (Ng et al, 2013).…”

“…3). The approach presented in this work was evaluated empirically and competitively in the context of the CoNLL shared task on error correction (Ng et al, 2013) where it was implemented as part of the highest-scoring University of Illinois system and demonstrated superior performance on the verb error correction sub-task. This paper makes the following contributions:…”