Transcription of Spanish Historical Handwritten Documents with Deep Neural Networks

Abstract

Abstract:The digitization of historical handwritten document images is important for the preservation of cultural heritage. Moreover, the transcription of text images obtained from digitization is necessary to provide efficient information access to the content of these documents. Handwritten Text Recognition (HTR) has become an important research topic in the areas of image and computational language processing that allows us to obtain transcriptions from text images. State-of-the-art HTR systems are, however, far from perfect. One difficulty is that they have to cope with image noise and handwriting variability. Another difficulty is the presence of a large amount of Out-Of-Vocabulary (OOV) words in ancient historical texts. A solution to this problem is to use external lexical resources, but such resources might be scarce or unavailable given the nature and the age of such documents. This work proposes a solution to avoid this limitation. It consists of associating a powerful optical recognition system that will cope with image noise and variability, with a language model based on sub-lexical units that will model OOV words. Such a language modeling approach reduces the size of the lexicon while increasing the lexicon coverage. Experiments are first conducted on the publicly available Rodrigo dataset, which contains the digitization of an ancient Spanish manuscript, with a recognizer based on Hidden Markov Models (HMMs). They show that sub-lexical units outperform word units in terms of Word Error Rate (WER), Character Error Rate (CER) and OOV word accuracy rate. This approach is then applied to deep net classifiers, namely Bi-directional Long-Short Term Memory (BLSTMs) and Convolutional Recurrent Neural Nets (CRNNs). Results show that CRNNs outperform HMMs and BLSTMs, reaching the lowest WER and CER for this image dataset and significantly improving OOV recognition.

A set of benchmarks for Handwritten Text Recognition on historical documents

“…Details can be seen in the scripts provided for this specific dataset in GitHub. 10 Given the relatively scarce text data available in the training transcripts, a character 7-gram was used in this case.…”

“…Full architecture and training details are included in the scripts available for each dataset in GitHub. 10 As previously discussed, for a given text line image, a trained CRNN estimates a sequence of character posterior probability vectors (often referred to as "ConfMat"). While raw images can be directly accepted as input, results can often be improved if images are previously deskewed, deslanted, cleaned, contrast-enhanced, and/or size-normalized [15,16,17,18].…”

“…It is completely written in old Castilian (Spanish) by a single author. The best WER for the evaluation partition in this dataset is 14% [10]. A historical Arabic dataset is referred to in [11], but it is quite recent and no WER results are provided in the reference.…”

Separating Optical and Language Models Through Encoder-Decoder Strategy for Transferable Handwriting Recognition

“…State of the art handwriting systems are built with a Multidimensional Long Short-Term Memory (MDLSTM) network [3] or even with a Convolutional Recurrent Neural Network (CRNN) stacked with a Bidirectional Long Short-Term Memory (BLSTM) [4]. The neural network training includes a Connectionist Temporal Classification (CTC) cost function proposed by [5].…”

Document Image Processing

“…Several binarization and text-line segmentation approaches are also benchmarked on these specific documents. The work by Granell et al [9] describes an efficient text-line recognition system, based on CNN and stacks of RNNs, that has been developed for the recognition of historical Spanish documents. These documents include out-of-vocabulary ancient words which are handled by a language model based on sub-lexical units.…”