Text Line Extraction in Historical Documents Using Mask R-CNN

Abstract

Text line extraction is an essential preprocessing step in many handwritten document image analysis tasks. It includes detecting text lines in a document image and segmenting the regions of each detected line. Deep learning-based methods are frequently used for text line detection. However, only a limited number of methods tackle the problems of detection and segmentation together. This paper proposes a holistic method that applies Mask R-CNN for text line extraction. A Mask R-CNN model is trained to extract text lines fractions from document patches, which are further merged to form the text lines of an entire page. The presented method was evaluated on the two well-known datasets of historical documents, DIVA-HisDB and ICDAR 2015-HTR, and achieved state-of-the-art results. In addition, we introduce a new challenging dataset of Arabic historical manuscripts, VML-AHTE, where numerous diacritics are present. We show that the presented Mask R-CNN-based method can successfully segment text lines, even in such a challenging scenario.

Historical Text Line Segmentation Using Deep Learning Algorithms: Mask-RCNN against U-Net Networks

“…In fact, the other study we know of, where Mask-RCNN was used for text line segmentation in historical documents with Latin-based or Arabic scripts [ 16 ], gives much more encouraging results. In this study, the network was trained to segment fragments of text lines into overlapping patches, which are then merged together to reconstruct the entire page with its multiple segmented text lines.…”

“…These encouraging results confirm our preliminary data showing that Mask-RCNN is a promising network for text line segmentation of historical documents, at least with Western scripts [ 30 , 31 ]. In parallel with [ 16 ], we actually developed another approach to exploit the Mask-RCNN architecture for the same purpose, while overcoming some of the weaknesses of their own approach. Indeed, the networks with which these authors challenged Mask-RCNN are not recent ones and, unlike U-Net networks, cannot be considered as state-of-the-art artificial neural networks.…”

“…Mask-RCNN belongs to the first category. We present an original way of exploiting the Mask-RCNN architecture, which differs from the main work we currently know that uses this network to extract lines of text in historical documents [ 16 ]. We will first describe the U-Net and Mask-RCNN networks and introduce the work of [ 16 ], which allows us to give the rationale for our choice to select the study of [ 17 ] as the source of our performance comparisons.…”

“…We present an original way of exploiting the Mask-RCNN architecture, which differs from the main work we currently know that uses this network to extract lines of text in historical documents [ 16 ]. We will first describe the U-Net and Mask-RCNN networks and introduce the work of [ 16 ], which allows us to give the rationale for our choice to select the study of [ 17 ] as the source of our performance comparisons. U-Net and Mask-RCNN networks are then compared in the context of text line segmentation for text transcription using different public databases (cBaD 2017 READ-Complex [ 4 ], DIVA-HisDB [ 18 ], HOME-Alcar [ 19 ]) and a private database (the Deliberation Registers of the States of Burgundy, hereafter DRoSB).…”

Text line and word detection and recognition of historical Arabic manuscripts

“…These models based Fully Convolutional Networks (FCN) are trained to detect the baseline or x-heights of text line which are not enough to extract a complete line. Recently (Droby et al, 2022) used mask R-CNN to extract lines of text from the entire page by combining the results from the segmentation of small patches of the page document. They reaches 87.96% of F-measure on 10 pages of the VML-AHTE dataset with post-processing that refines the extraction of lines of text.…”