Measuring the Correctness of Double-Keying: Error Classification and Quality Control in a Large Corpus of TEI-Annotated Historical Text

Abstract

Among mass digitization methods, double-keying is considered to be the one with the lowest error rate. This method requires two independent transcriptions of a text by two different operators. It is particularly well suited to historical texts, which often exhibit deficiencies like poor master copies or other difficulties such as spelling variation or complex text structures.Providers of data entry services using the double-keying method generally advertise very high accuracy rates (around 99.95% to 99.98%). These advertised percentages are generally estimated on the basis of small samples, and little if anything is said about either the actual amount of text or the text genres which have been proofread, about error types, proofreaders, etc. In order to obtain significant data on this problem it is necessary to analyze a large amount of text representing a balanced sample of different text types, to distinguish the structural XML/TEI level from the typographical level, and to differentiate between various types of errors which may originate from different sources and may not be equally severe. This paper presents an extensive and complex approach to the analysis and correction of doublekeying errors which has been applied by the DFG-funded project "Deutsches Textarchiv" (German Text Archive, hereafter DTA) in order to evaluate and preferably to increase the transcription and annotation accuracy of double-keyed DTA texts. Statistical analyses of the results gained from proofreading a large quantity of text are presented, which verify the common accuracy rates for the double-keying method.

Task design and assignment of full-text generation on mass Chinese historical archives in digital humanities

“…The two resulting versions are compared in order to detect transcription errors. The premise of double-keying is that two human operators are unlikely to make the same mistakes (Haaf et al , 2013). However, because, as we have already established, Chinese characters are so diverse and variable, two independent operators in a double-keying process may recognize the same form as different characters and thus may produce two results, which may both be right or both be wrong.…”

Population as Auditor of an Election Process in Honduras: The Case of the VotoSocial Crowdsourcing Platform

“…According to Doan et al () VotoSocial's architecture is explicit; that is, users explicitly collaborate with the platform, as opposed to implicit collaboration where users collaborate as a side effect of their actions in a system. Furthermore, VotoSocial assigns the users with task execution, all work is undertaken online, and unlike other systems that send tickets when a user finds a mis‐transcription (Haaf, Wiegand, & Geyken, ), the correction is done online and almost in real time. Still, users may send reports when they find suspicious records such as the one seen in Figure .…”

“…A triple verification process was used, allowing for user‐controlled validation of the data. This is similar to a double‐keying system (Haaf et al, ), but it was decided that triple validation would be more user friendly (i.e., requiring only a single click to register a transcription as accurate or not; as opposed to double‐keying systems where all users need to input the entire text that is being digitized) without compromising accuracy, and would also enable the community to automatically validate the digitization process. (See Figure , bottom left, showing the two buttons to register the transcription as either correct or incorrect.)…”

Correcting Whitespace Errors in Digitized Historical Texts

“…Alternatively, with substantial effort, digitization can be performed manually, or by manual correction of OCR output (Tanner et al, 2009). However, even for manually "keyed-in" corpora, noise can be introduced due to errors in workflow (Haaf et al, 2013).…”