Syntactic Annotation of a German Newspaper Corpus

Brants, Thorsten; Skut, Wojciech; Uszkoreit, Hans

doi:10.1007/978-94-010-0201-1_5

Cited by 50 publications

(44 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, researchers advocate that consistency highly increases the usefulness of a corpus for training or evaluation purposes, and it is crucial to the success of machine learning algorithms [2] [29]. Therefore, studies have been conducted to research scientific methodology for creating high quality annotations, addressing inter-annotator consistency.…”

Section: Annotator Discrepancymentioning

confidence: 99%

“…For example, Brants [2] reports their work on creating syntactic annotations (partof-speech and structural information) on a German newspaper corpus. The activity involves trained annotators performing the annotation tasks at sentence level independently, then cross-checking and discussing together to resolve discrepancies.…”

Section: How To Annotate Properly: What Have We Learnt?mentioning

confidence: 99%

“…The process is repeated as much as possible until the level of discrepancies is reduced to a satisfactory level. Such a repetitive process often requires months and even up to years of work from experienced researchers [2] [29], yet discrepancies can never be eliminated [14] and the resulting annotations and guidelines are often application-specific and nongeneralisable. Given such an ineffective and inefficient process, the tremendous cost from key experts means that it is inapplicable in many practical situations such as industries, due to resource limitations (e.g., finance, time and personnel) [15].…”

mentioning

confidence: 99%

See 2 more Smart Citations

A Methodology towards Effective and Efficient Manual Document Annotation: Addressing Annotator Discrepancy and Annotation Quality

Zhang

Chapman

Ciravegna

2010

Knowledge Engineering and Management by the Masses

View full text Add to dashboard Cite

Abstract. Manual document annotation is an essential technique for knowledge acquisition and capture. Creating high-quality annotations is a difficult task due to inter-annotator discrepancy, the problem that annotators can never agree completely on what and exactly how to annotate. To address this, traditional document annotation involves multiple domain experts working on the same annotation task in an iterative and collaborative manner to identify and resolve discrepancies progressively. However, such a detailed process is often ineffective despite taking significant time and effort; unfortunately, discrepancies remain high in many cases. This paper proposes an alternative approach to document annotation. The approach tackles the problem by firstly studying annotators' suitability based on the types of information to be annotated; then identifying and isolating the most inconsistent annotators who tend to cause the majority of discrepancies in a task; finally distributing annotation workload among the most suitable annotators. Tested in a named entity annotation task in the domain of archaeology, we show that compared to the traditional approach to document annotation, it produces larger amounts of better quality annotations that result in higher machine learning accuracy while requires significantly less time and effort.

show abstract

Section: Annotator Discrepancymentioning

confidence: 99%

Section: How To Annotate Properly: What Have We Learnt?mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

A Methodology towards Effective and Efficient Manual Document Annotation: Addressing Annotator Discrepancy and Annotation Quality

Zhang

Chapman

Ciravegna

2010

Knowledge Engineering and Management by the Masses

View full text Add to dashboard Cite

show abstract

“…The cost of syntactic annotation of text (PoS tagging and annotating syntactic structure and labels for nodes and edges) may run to an average of 50 seconds per sentence with an average sentence length of 17.5 tokens (cf. Brants et al [30], which describes syntactic annotation of a German newspaper corpus). As a final example, Lin et al [31] report an annotation efficiency of 6.8xRT for annotating MPEG-7 metadata on video using the VideoAnnEx tool.…”

Section: Manual Annotationsmentioning

confidence: 99%

Meeting Modelling in the Context of Multimodal Research

Reidsma

Rienks

Jovanović

2005

Machine Learning for Multimodal Interaction

View full text Add to dashboard Cite

Abstract. This paper presents a framework for corpus based multimodal research. Part of this framework is applied in the context of meeting modelling. A generic model for different aspects of meetings is discussed. This model leads to a layered description of meetings where each layer adds a level of interpretation for distinct aspects based on information provided by lower layers. This model should provide a starting point for selecting annotation schemes for layers of the meeting and for defining a hierarchy between individual layers.

show abstract

“…The branches of such trees may cross in order to treat non-local dependencies. Negra comes with a tool that transforms the Negra-format to the Penn-format by transforming crossing edges into non-crossing edges and by the introduction of corresponding gap nodes, Skut et al, 1997;Brants et al, 2000. We are using these transformed Negra-trees in our experiments.…”

Section: Sltg From Treebanksmentioning

confidence: 99%

A Uniform Method for Automatically Extracting Stochastic Lexicalized Tree Grammars from Treebanks and HPSG

Neumann¹

2003

Treebanks

View full text Add to dashboard Cite

We present a uniform method for the extraction of stochastic lexicalized tree grammars (SLTG) of different complexities from existing treebanks as well as from competence-based grammars , which allows us to analyze the relationship of a grammar automatically induced from a treebank with respect to its size, its complexity, and its predictive power on unseen data. Processing of different SLTG is performed by a stochastic version of the two-step Early-based parsing strategy introduced in Schabes and Joshi, 1991.

show abstract

Syntactic Annotation of a German Newspaper Corpus

Cited by 50 publications

References 10 publications

A Methodology towards Effective and Efficient Manual Document Annotation: Addressing Annotator Discrepancy and Annotation Quality

A Methodology towards Effective and Efficient Manual Document Annotation: Addressing Annotator Discrepancy and Annotation Quality

Meeting Modelling in the Context of Multimodal Research

A Uniform Method for Automatically Extracting Stochastic Lexicalized Tree Grammars from Treebanks and HPSG

Contact Info

Product

Resources

About