A prototype gutenberg-hathitrust sentence-level parallel corpus for OCR error analysis

Abstract

This exploratory study proposes a prototype sentence-level parallel corpus to support studying optical character recognition (OCR) quality in curated digitized library collections. Existing data resources, such as ICDAR2019 [21] and GT4HistOCR[23], generally aligned content by artifact publishing characteristics such as documents or lines, which is limited to explore OCR noise concentrating on natural language granularity like sentences and chapters. Building upon an existing volume-aligned corpus that collected human-proofread texts from Project Gutenberg and paired OCR views from HathiTrust Digital Library, we extracted and aligned 167,079 sentences from 189 sampled books in four domains published from 1793 to 1984. To support downstream research on OCR quality, we conducted an analysis of OCR errors with a specific focus on their associations with the source text metadata. We found that sampled data in agriculture has a higher ratio of real-word errors than other domains, while sentences from social-science volumes contain more non-word errors. Besides, data sampled from early-age volumes tend to have a high ratio of non-word errors, while samples from recently-published volumes is likely to have more real-word errors. Following our findings, we suggest that scholars should consider the potential influence of source data characteristics on their findings in the study of OCR quality issues. CCS CONCEPTS• Information systems → Digital libraries and archives; • Applied computing → Document management and text processing; Document capture.

Evaluating BERT-based scientific relation classifiers for scholarly knowledge graph construction on digital library collections

“…With the increasing popularity of applying NLP techniques to DL textual resources for macro-level computation research [24,4], concerns about the reliability of NLP techniques for processing digitized library collections have recently been on the rise [27,45,48]. Based on our literature review, one of the major issues that challenges NLP techniques' reliability on OCR'd texts is their potential inclusion of errors resulting from the OCR process [27,45,48].…”

“…Overall, existing work concentrating on the investigation of the impact of OCR errors on NLP tasks can be divided into two groups. One is based on quantitative analysis [27,45]. In this group, researchers usually measure and compare the performance differences of the same NLP tool applied on the clean versus the OCR'd version of texts.…”