The DTA “Base Format”: A TEI Subset for the Compilation of a Large Reference Corpus of Printed Text from Multiple Sources

Abstract

Until recently the creation of large historical reference corpora was, from the point of view of its encoding, a rather project-specic activity. Although reference corpora were built from texts of various origins, the texts had to be converted into a tailor-made format. For example, corpora like the well-known British National Corpus 1 and the DWDS core corpus (Geyken 2007) are both annotated on the basis of the Guidelines of the Text Encoding Initiative (most recent release: P5; see TEI Consortium 2014). However, the encoding in these cases was typically carried out specically for the creation of these corpora-that is, it was a unidirectional process. The interchange, and more importantly the interoperability, of corpora with other corpora played only a minor role. 2 In recent years the picture has dramatically changed. With the availability of more and more digitized texts in TEI P5 format and the advent of many academic and non-academic corpusbuilding projects, the task of creating reference corpora has shifted from a project-specic task to a more general task, requiring joint eorts by many stakeholders. In this new situation, individual

Digitizing University Libraries – Evolving from Full-Text Providers to CLARIN Contact Points on Campuses

“…The resulting OCR full text was processed by the OCR software ABBYY Finereader 9 and consists of approximately 500 million characters and 65 million tokens. As a second aspect of text quality, we enhanced the level of document structure according to an agreed standard format together with our partners, the Deutsches Textarchiv (DTA; Haaf Geyken and Wiegand, 2014/15). Figure 1 shows manually corrected and tagged "zoning information" based on coordinates provided by the ABBYY Finereader XML files.…”

“…There has to be a decision for ALTO, PAGE, TEI or other file formats, possibly together with 'annotation guidelines'. (Haaf et al, 2014/15) Supplemental note: The use of format converters should be considered with care. There will always be a loss of information converting from a format to another.…”

Metrical Tagging in the Wild: Building and Annotating Poetry Corpora with Rhythmic Features

“…The .json format is intended for ease of use and speed of processing while retaining some expressiveness. Our XML format is built on top of a "Base Format", the socalled DTA-Basisformat 3 (Haaf et al, 2014) that not only constrains the data to TEI P5 guidelines, but also regarding a stricter relaxNG schema that we modified for our annotation. 4 We built a large, comprehensive, and easily searchable resource of New High German poetry by collecting and parsing the bulk of digitized corpora that contain public domain German literature.…”

Corpus17

“…To follow our logic of interoperability, like many other literary corpora, we have decided to encode our corpus in XML-TEI P5 [Burnard 2014]. Because documenting the encoding choices is (sadly according to Burnard [2019]) not common in France, our decisions are inspired by two non-French projects: the Deutsches Textarchiv (DTA) [Haaf et al 2014] Table 5: Lemmatisation accuracies of the best model on outof-domain data.…”