Transcribing a 17th-century botanical manuscript: Longitudinal evaluation of document layout detection and interactive transcription

Abstract

We present a process for cost-effective transcription of cursive handwritten text images that has been tested on a 1 000 pages 17th century book about botanical species. The process comprised two main tasks, namely: (1) preprocessing: page layout analysis, text line detection, and extraction; and (2) transcription of the extracted text line images. Both tasks were carried out with semiautomatic procedures, aimed at incrementally minimizing user correction effort, by means of computer-assisted line detection and interactive handwritten text recognition technologies. The contribution derived from this work is threefold. First, we provide a detailed humansupervised transcription of a relatively large historical handwritten book, ready to be searchable, indexable, and accessible to cultural heritage scholars as well as the general public. Second, we have conducted the first longitudinal study to date on interactive handwriting text recognition, for which we provide a very comprehensive user assessment of the real-world performance of the technologies involved in this work. Third, as a result of this process, we have produced a detailed transcription and document layout information (i.e., high-quality labeled data) ready to be used by researchers working on automated technologies for document analysis and recognition.

One Step Is Not Enough: A Multi-Step Procedure for Building the Training Set of a Query by String Keyword Spotting System to Assist the Transcription of Historical Document

“…The HTR system proposed a full transcript of a given text line image and every time the user amended a wrong word the following ones were updated by the system. In [ 46 ] the user was an expert palaeographer while in [ 47 ] students in History were involved. In both the papers, the authors noticed a significant typing effort reduction that did not result in a net user time effort savings.…”

“…Therefore, Table 6 provides a rough indication of the transcription time spent by the palaeographers that adopted one of the listed systems. 4 52.54% The experimental studies presented in [46,47] were performed with the same HTR, which is based on Hidden Markov modes and N-grams models, on two different document collections. The HTR system proposed a full transcript of a given text line image and every time the user amended a wrong word the following ones were updated by the system.…”

Advances in Handwritten Keyword Indexing and Search Technologies

“…An alternative to fully automatic processing is to rely on computer-assisted transcription. This was successfully explored empirically by Toselli et al (2017), Romero et al(2012) and Alabau et al (2014), following new, powerful concepts of pattern recognition-based human-machine interaction introduced by Vidal et al (2007) and Toselli et al (2011). Following the positive results of these laboratory studies, preliminary evaluation by real users was carried out by Toselli et al (2016).…”

A Probabilistic Formulation of Keyword Spotting

“…We use two large data sets, from the Bentham and Plantas collections. The Bentham partition that we use is the one used in the ICFHR2014 Competition on Handwritten Text Recognition on Transcriptorium Datasets [Sánchez et al, 2014], while the Plantas partition was introduced in [Toselli et al, 2018a]. More details about these databases can be found in appendices A.1 and A.5, respectively.…”

“…We excluded empty page images or those containing only drawings. Full details about this database are found in [Toselli et al, 2018a]. It is important to mention that reference transcripts of Plantas are provided in two different versions: diplomatic and modernized.…”