Detecting Text in Natural Image with Connectionist Text Proposal Network

Abstract

Abstract. We propose a novel Connectionist Text Proposal Network (CTPN) that accurately localizes text lines in natural image. The CTPN detects a text line in a sequence of fine-scale text proposals directly in convolutional feature maps. We develop a vertical anchor mechanism that jointly predicts location and text/non-text score of each fixed-width proposal, considerably improving localization accuracy. The sequential proposals are naturally connected by a recurrent neural network, which is seamlessly incorporated into the convolutional network, resulting in an end-to-end trainable model. This allows the CTPN to explore rich context information of image, making it powerful to detect extremely ambiguous text. The CTPN works reliably on multi-scale and multilanguage text without further post-processing, departing from previous bottom-up methods requiring multi-step post filtering. It achieves 0.88 and 0.61 F-measure on the ICDAR 2013 and 2015 benchmarks, surpassing recent results [8,35] by a large margin. The CTPN is computationally efficient with 0.14s/image, by using the very deep VGG16 model [27]. Online demo is available at: http://textdet.com/.

Detecting Oriented Text in Natural Images by Linking Segments

“…Text Detection Over the past few years, much research effort has been devoted to the text detection problem [24,23,17,17,25,7,8,30,29,2,9,6,22,26]. Based on the basic detection targets, the previous methods can be roughly divided into three categories: character-based, word-based and line-based.…”

“…The two protocols usually yield very close scores. [29] 88 74 80 <0.1 Zhang et al [30] 88 78 83 <1 Jaderberg et al [9] 88.5 67.8 76.8 <1 Gupta et al [6] 92.0 75.5 83.0 15 Tian et al [22] 93.0 83.0 87.7 7.1 SegLink 87.7 83.0 85.3 20.6…”

Chinese text-line detection from web videos with fully convolutional networks

“…In [12], Jaderberg et al proposed a text detection and recognition system based on region proposal mechanism and deep CNN. The Connectionist Text Proposal Network (CTPN) [8] innovatively combined Faster R-CNN with LSTM and constructed a bottom-to-up model to detect horizontal text lines. Inspired by Faster R-CNN, Ma et al [13] introduced a novel rotated anchors based framework for arbitrary-oriented text detection.…”

“…For mainly two reasons, however, it is difficult to apply these general object detection systems directly to caption detection, which generally requires a higher localization accuracy. Firstly, in generic object detection, each object has a welldefined closed boundary, while such a well-defined boundary may not exist in caption text, since a text line or word is composed of a number of separate characters or strokes [8]. Secondly, text has a larger aspect ratio range than generic objects that text could be short or long in different directions.…”

Scene Text Detection based on Enhanced Multi-channels MSER and a Fast Text Grouping Process

“…Hybrid method [18][19][20] takes advantage of both texture-based methods and component-based ones. Fabrizio et al [19] proposed a hybrid and multi-scale text detection algorithm that can better handle "challenging text" such as multi-size, multi-color and multi-orientation; but these methods are time consuming and need a pre-set lexicon for every image.…”