Boosting Offline Handwritten Text Recognition in Historical Documents With Few Labeled Lines

Abstract

In this paper we address the problem of offline handwritten text recognition (HTR) in historical documents when few labeled samples are available and some of them contain errors in the train set. Our three main contributions are: first, we analyze how to perform transfer learning (TL) from a massive database to a smaller historical database, analyzing which layers of the model need fine-tuning. Second, we analyze methods to efficiently combine TL and data augmentation (DA). Finally, we propose an algorithm to mitigate the effects of incorrect labeling in the training set. The methods are analyzed over the ICFHR 2018 competition database, Washington and Parzival. Combining all these techniques, we demonstrate a remarkable reduction of CER (up to 6 percentage points in some cases) in the test set with little complexity overhead.INDEX TERMS connectionist temporal classification (CTC), convolutional neural networks (CNN), data augmentation (DA), deep neural networks (DNN), historical documents, long-short-term-memory (LSTM), offline handwriting text recognition (HTR), outlier detection; transfer learning.

Handwritten text generation and strikethrough characters augmentation

“…For real HTR tasks, it is not essential to track the case of characters. We compare our approach on a few labeled datasets (Konzil, Schiller, Ricordi, Patzig, Schwerin) with results from paper [11]. Results are presented in Tab.…”

“…6. For our model, we took the best results from Table 4, and the best results from Table 1 in paper [11]. Our model achieves better CER for 3 of 5 datasets.…”

“…Our model achieves better CER for 3 of 5 datasets. Additionally, we did not use transfer learning from bigger datasets, unlike authors [11]. Data that we used for Table 6 are located in "specif-ic_data" folder.…”

“…So, for the test set in Table 4, we took all files marked with "train_4" for each dataset. But authors of [11] used the test set (with ground truth) for ICFHR 2018 Competition [44], which is hidden for us. Despite the usage of different test sets, we think this comparison is relatively fair.…”

“…Common tricks may significantly improve the quality of HTR models. In [11], the authors investigated data augmentation and transfer learning for small historical datasets.…”