A survey of historical document image datasets

Abstract

This paper presents a systematic literature review of image datasets for document image analysis, focusing on historical documents, such as handwritten manuscripts and early prints. Finding appropriate datasets for historical document analysis is a crucial prerequisite to facilitate research using different machine learning algorithms. However, because of the very large variety of the actual data (e.g., scripts, tasks, dates, support systems, and amount of deterioration), the different formats for data and label representation, and the different evaluation processes and benchmarks, finding appropriate datasets is a difficult task. This work fills this gap, presenting a meta-study on existing datasets. After a systematic selection process (according to PRISMA guidelines), we select 65 studies that are chosen based on different factors, such as the year of publication, number of methods implemented in the article, reliability of the chosen algorithms, dataset size, and journal outlet. We summarize each study by assigning it to one of three pre-defined tasks: document classification, layout structure, or content analysis. We present the statistics, document type, language, tasks, input visual aspects, and ground truth information for every dataset. In addition, we provide the benchmark tasks and results from these papers or recent competitions. We further discuss gaps and challenges in this domain. We advocate for providing conversion tools to common formats (e.g., COCO format for computer vision tasks) and always providing a set of evaluation metrics, instead of just one, to make results comparable across studies.

FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning

“…LDM (Rombach et al 2022) proposes a cross-attention mechanism to incorporate the condition into the UNet and treats the diffusion process in the latent space. In text image generation, (Luhman and Luhman 2020;Gui et al 2023;Nikolaidou et al 2023) apply diffusion models to generate handwritten characters and demonstrate their promising effects. CTIG-DM (Zhu et al 2023) devises image, text, and style as conditions and introduces four text image generation modes in a diffusion model.…”

CorDeep and the Sacrobosco Dataset: Detection of Visual Elements in Historical Documents

“…While page segmentation approaches, such as [ 7 ], return text/non-text masks, the latter of which often includes visual elements, this remains insufficient for any meaningful historical study, as such approaches often lack accurate visual element localization, as well as semantic classification of these elements. In this regard, one of the main hurdles hindering the success of semantic visual element recognition within historical documents is their high variability, as well as the general scarcity of coherent historical datasets focused on visual element recognition, with only 11 out of the 56 historical document datasets mentioned in [ 15 ] containing graphical elements. Additionally, in the majority of cases where visual elements were recorded in these datasets, they were not classified according to their semantic classes [ 16 ].…”

Detecting Tool Keypoints with Synthetic Training Data

“…In contrast, modern learning-based methods exhibit the ability to infer styled glyphs, even in cases where they have not been directly observed in the reference style examples. Despite a few attempts to perform HTG with Diffusion Models [33], [34], the most typical strategy is to leverage GANs, which can be unconditioned in the case of non-stylized HTG or conditioned on a variable number of handwriting style samples in the case of stylized HTG.…”

“…However, there is no agreement on the split to adopt, i.e., on which authors and relative images should be included in the training set and which in the test. As a result, some works adopt the standard HTR split [1], [3], [11], [25], [34], [47], [48] (commonly known as Aachen split), while others [2], [24], [26], [27], [29], [33], [49]- [51] consider the original split proposed with the IAM dataset, which entails a different distribution of the authors between training and test. Moreover, the text content of the generated words and the style samples considered for each author in styled-HTG are usually selected randomly, thus further hindering the fair comparison also between approaches adopting the same IAM splitting.…”

“…The resulting textual content representation is then combined with the style features and fed to another model component that outputs the styled text image. For training HTG models, most of the State-of-the-Art approaches (apart from a few preliminary works exploiting Diffusion Models [33], [34]) follow the adversarial learning paradigm. This means that the HTG model is the generator, and another dedicated network is used as a discriminator to distinguish the writer's real images from the model's generated ones.…”