Competing target hypotheses in the Falko corpus

Reznicek, Marc; Lüdeling, Anke; Hirschmann, Hagen

doi:10.1075/scl.59.07rez

Cited by 25 publications

(16 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Rosen et al (2014) (see also its discussion in Meurers, 2015) have shown that differing target hypotheses among annotators account for a considerable amount of disagreement in the choice of error tags. They conclude, in line with Reznicek et al (2013), that an explicit target hypothesis is required for annotating learner errors. While target hypotheses in general are said to be hard to agree on (Lüdeling, 2008;Fitzpatrick and Seegmiller, 2004), minimal target hypotheses, i.e.…”

Section: Introductionmentioning

confidence: 65%

“…Two German L2 learner corpora are annotated with more than one target hypothesis: Falko (Reznicek et al, 2012) and EAGLE (Boyd, 2010). Falko treats orthographic and grammatical errors together at the first layer, though, and semantic/stylistic errors on the second.…”

Section: Related Workmentioning

confidence: 99%

“…This has the advantage that errors do not have to be cast into pre-defined categories, which might not capture all cases (Fitzpatrick and Seegmiller, 2004). However, as Reznicek et al (2013) demonstrate, there is a possibly infinite number of target hypotheses for a single utterance, depending on the linguistic level that is corrected (orthography, grammar, lexis, etc.). They argue further that the usefulness of a target hypothesis depends on the research purpose, and that its construction must be comprehensible and transparent to other researchers.…”

Section: Introductionmentioning

confidence: 99%

“…The identification and annotation of errors necessarily depends on a target hypothesis, i.e. the assumed correct form of the learner's utterance, be that stated implicitly or explicitly (Reznicek et al, 2013). The correct form itself can already serve as error annotation.…”

Section: Introductionmentioning

confidence: 99%

“…minimal form changes that are required to make an utterance grammatical (Meurers, 2015), are generally presented as less problematic for inter-annotator agreement (see e.g. Reznicek et al (2012) on the minimal target hypothesis in the Falko corpus). However, we are not aware of a study which systematically evaluates the agreement on such a minimal target hypothesis in a corpus.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Annotating Orthographic Target Hypotheses in a German L1 Learner Corpus

Laarmann-Quante

Ortmann

Ehlert

et al. 2017

Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications

View full text Add to dashboard Cite

NLP applications for learners often rely on annotated learner corpora. Thereby, it is important that the annotations are both meaningful for the task, and consistent and reliable. We present a new longitudinal L1 learner corpus for German (handwritten texts collected in grade 2-4), which is transcribed and annotated with a target hypothesis that strictly only corrects orthographic errors, and is thereby tailored to research and tool development for orthographic issues in primary school. While for most corpora, transcription and target hypothesis are not evaluated, we conducted a detailed inter-annotator agreement study for both tasks. Although we achieved high agreement, our discussion of cases of disagreement shows that even with detailed guidelines, annotators differ here and there for different reasons, which should also be considered when working with transcriptions and target hypotheses of other corpora, especially if no explicit guidelines for their construction are known.

show abstract

Section: Introductionmentioning

confidence: 65%

Section: Related Workmentioning

confidence: 99%