A formal framework for linguistic annotation

Abstract

Abstract'Linguistic annotation' covers any descriptive or analytic notations applied to raw language data. The basic data may be in the form of time functions -audio, video and/or physiological recordings -or it may be textual. The added notations may include transcriptions of all sorts (from phonetic features to discourse structures), part-of-speech and sense tagging, syntactic analysis, 'named entity' identification, co-reference annotation, and so on. While there are several ongoing efforts to provide formats and tools for such annotations and to publish annotated linguistic databases, the lack of widely accepted standards is becoming a critical problem. Proposed standards, to the extent they exist, have focussed on file formats. This paper focuses instead on the logical structure of linguistic annotations. We survey a wide variety of existing annotation formats and demonstrate a common conceptual core, the annotation graph. This provides a formal framework for constructing, maintaining and searching linguistic annotations, while remaining consistent with many alternative data structures and file formats.

Linguistic Annotation in/for Corpus Linguistics

“…Multimodal corpora present challenges for mapping layers of annotation to time series data like audio and video recordings. Bird and Liberman [10] present a model for the logical structure of layers of annotation and time known as an annotation graph. An annotation graph allows for the flexible establishment of a hierarchical series of annotation nodes with a fundamental node based on either character position for text corpora or time offsets for speech corpora.…”

An Exchange Format for Multimodal Annotations

“…All formats have in common that their basic building blocks are annotation tuples consisting of a start and an end point (with, typically, a temporal interpretation) and one or more text labels (with no fixed interpretation). Since this is precisely the principle on which the annotation graph formalism (AG, see [4]) is based, it was natural to choose AGs as a general framework for our task. As a further advantage, several software packages facilitating the work with AGs already exist [2].…”

“…Instead, they inherit their start and end points from the corresponding annotation in the parent tier. 4 This increases robustness since a misalignment of such logically related elements can never occur. Apart from such structural constraint on the tiers' content, ANVIL also allows to organize tiers themselves hierarchically into groups and subgroups for visual organization (like directory folders) and inheritance of coding scheme properties (group attributes are inherited by all contained tracks).…”

Tools to address the interdependence between tokenisation and standoff annotation

“…There are a number of decision points involved in the development of such tools, some of which come down to a matter of preference and some of which are consequences of other choices. Examples of annotation methods which are not primarily based on XML are GATE (Cunningham et al, 2002) and the annotation graph model of Bird and Liberman (2001). The GATE system organises annotations in graphs where the start and end nodes have pointers into the source document character offsets.…”