dhSegment: A Generic Deep-Learning Approach for Document Segmentation

Abstract

In recent years there have been multiple successful attempts tackling document processing problems separately by designing task specific hand-tuned strategies. We argue that the diversity of historical document processing tasks prohibits to solve them one at a time and shows a need for designing generic approaches in order to handle the variability of historical series. In this paper, we address multiple tasks simultaneously such as page extraction, baseline extraction, layout analysis or multiple typologies of illustrations and photograph extraction.We propose an open-source implementation of a CNN-based pixel-wise predictor coupled with task dependent post-processing blocks. We show that a single CNN-architecture can be used across tasks with competitive results. Moreover most of the task-specific post-precessing steps can be decomposed in a small number of simple and standard reusable operations, adding to the flexibility of our approach.

Multimodal Deep Networks for Text and Image-Based Document Classification

“…In the same idea, [12] also investigated such deep architectures to classify identity documents. [13] goes even further by trying to segment the full layout of a document image into paragraphs, titles, ornaments, images etc. These models focus on extracting strong visual features from the images to classify the documents based on their layout, geometry, colors and shape.…”

“…For industrial-grade applications dealing with user-generated content, such a data augmentation is necessary to alleviate overfitting and reduce the gap between train and actual data. Preprocessing page segmentation and layout analysis tools, such as dhSegment [13] can also bring significant improvements by renormalizing image orientation and cropping the document before sending it to the classifier. Moreover, as we have seen, the post-OCR word embeddings include lots of noisy or completely wrong words that generate OOV errors.…”

Advances in Document Layout Analysis

“…The learning problem is how to adjust the HMM parameters (a ij , b i (x), c jg , µ jg and Σ jg ), so that a given set of observations (called training set) is generated by the model with maximum likelihood. The Baum-Welch algorithm [20] (also known as Forward-Backward algorithm), is used to find these unknown parameters. It is an expectation-maximization (EM) algorithm.…”

“…Throughout the years several surveys have been performed [8,13,14,19,20,24] to compile this work and classify the underlying strategies used to tackle this task. I do not intend to provide a survey as complex and detailed as the aforementioned articles.…”

“…If we look at the most current works about convolutional neural networks (CNN) [20,22] applied to the TRDC task, we can observe that these networks are often used to produce "simplified images". These simplified images eliminate all types of noise present in the image and mark areas where the desired regions are present with some likelihood.…”