Text Line Extraction Using Fully Convolutional Network and Energy Minimization

Abstract

Text lines are important parts of handwritten document images and easier to analyze by further applications. Despite recent progress in text line detection, text line extraction from a handwritten document remains an unsolved task. This paper proposes to use a fully convolutional network for text line detection and energy minimization for text line extraction. Detected text lines are represented by blob lines that strike through the text lines. These blob lines assist an energy function for text line extraction. The detection stage can locate arbitrarily oriented text lines.Furthermore, the extraction stage is capable of finding out the pixels of text lines with various heights and interline proximity independent of their orientations. Besides, it can finely split the touching and overlapping text lines without an orientation assumption. We evaluate the proposed method on VML-AHTE, VML-MOC, and Diva-HisDB datasets. The VML-AHTE dataset contains overlapping, touching and close text lines with rich diacritics. The VML-MOC dataset is very challenging by its multiply oriented and skewed text lines. The Diva-HisDB dataset exhibits distinct text line heights and touching text lines. The results demonstrate the effectiveness of the method despite various types of challenges, yet using the same parameters in all the experiments.

Unsupervised Learning of Text Line Segmentation by Differentiating Coarse Patterns

“…We compare our results with those of supervised learning methods, Mask-RCNN [14] and FCN+EM [14], and an unsupervised deep learning method, UTLS [15]. Mask-RCNN is an instance segmentation algorithm which is fully supervised using the pixel labels of the text lines.…”

“…Foreground pixels definitely can not discriminate a text line from the others because FCN output is a semantic segmentation where multiple instances of the same object are not separated. Very recently, text line segmentation has been formulated as an instance segmentation problem using Mask-RCNN [10], and its results are available in [14]. However, when using FCN, each text line is represented as a single connected component.…”

“…However, when using FCN, each text line is represented as a single connected component. This component can be either a blob line [22,20,16,18,14] strikes through the main body area of the characters that belong to a text line or a baseline [9] passes through the bottom part of the main body of the characters that belong to a text line. FCNs are very successful at detecting handwritten text lines [7].…”

“…Alternatively, detected spatial location of text lines are utilized to further extract the pixels of text lines. Some of these extraction methods assume horizontal text lines [22] whereas some can extract text lines at any orientation, with any font type and font size [14]. Text line extraction is evaluated by classical image segmentation metrics [7].…”

“…The experiments cover five datasets that are different in terms of the challenges they pose. The VML-AHTE dataset [14] consists of Arabic handwritten documents with crowded diacritics and cramped text lines. The Pinkas dataset [3] contains slightly edge rounded and noisy images of Hebrew handwritten documents.…”