FTPN: Scene Text Detection With Feature Pyramid Based Text Proposal Network

Abstract

Scene text detection is to detect the position of a text in the natural scene, the quality of which will directly affect the subsequent text recognition. It plays an important role in fields such as image retrieval and autopilot. How to perform multi-scale and multi-oriented text detection in the scene still remains as a problem. This paper proposes an effective scene text detection method that combines the convolutional neural network (CNN) and recurrent neural network (RNN). In order to better adapt to texts in different scales, feature pyramid networks (FPN) have been applied in the CNN part to extract multi-scale features of the image. We then utilize bidirectional long-short-term memory (Bi-LSTM) to encode these features to make full use of the text sequence characteristics with the outputs as a series of text proposals. The generated proposals are finally linked into a text line through a well-designed text connector, which can be flexibly adapted to any oriented texts. The proposed method is evaluated on three public datasets: ICDAR2013, ICDAR2015, and USTB-SV1K. For ICDAR2013 and USTB-1K, we have reached 92.5% and 62.6% F-measure, respectively. Our method has reached 72.8% F-measure on the more challenging ICDAR2015 which demonstrates the effectiveness of our method. INDEX TERMS Scene text detection, multi-orientation, convolutional neural network, recurrent neural network, residual network.

Pelee-Text++: A Tiny Neural Network for Scene Text Detection

“…On ICDAR 2013, a dataset from natural scene images with presence of horizontal or near horizontal text, Pelee-Text++ reached competitive results with F-measure of 79.72% and 85.78% for its 768 and multi-scale versions, defeating Pelee-Text with a model 13 Megabytes lighter (see Table 7). Despite having a lower F-measure compared to state-of-theart methods, such as CRAFT [1], MaskTextSpotter [38], PMTD [32] and FTPN [31]; our proposal obtained a good trade-off between efficacy and model size. Figure 4a shows the balance between effectiveness and model size compared to state-of-the-art methods.…”

“…The proposed approaches have used different techniques for dealing with scene text detection, most of them are focused on regression [6,28,31,51,57], segmentation [4,8,56,56], or the combination of both techniques [35,38,39,52,63]. Additionally, some proposals have merged the detection and recognition tasks as a jointed pipeline for improving their results [34,38], or had even used knowledge distillation [68].…”

“…Most of those approaches are inspired by a wellknown object detector, Mask-RCNN [16]. In a general way, these methods [18,31] use a multi-scale feature extraction based on Feature Pyramid Networks (FPN) [30], or even 3D special pyramid mask for a better characterization of text instances [32]. Next, these features are fed to several branches, where each one of them has specific tasks, such as bounding box regression, text instance segmentation or refinement modules.…”

“…Despite the huge amount of methods developed to text detection proposed in the last years, most of these studies are focused on improving their results without concerns about their model size. Those approaches use VGG-16 [1,4,8,29,36,65,68] and ResNet [31,32,34,38,52,63] as feature extractors, producing models with a size ranging from 80 Megabytes [1] to more than 350 Megabytes [38]. For this reason, in practice, their use on devices with computational constraints turns non-viable.…”

Multi-Scale Geospatial Object Detection Based on Shallow-Deep Feature Extraction

“…The construction of the FPN aims to extract high-resolution and segmentation features by combining the output of the BU and TD pathways, but it takes a long time and consumes memory. At the same time, the development of computational processor devices like the graphics processing units (GPUs) have contributed to the improvement and development of image classification and recognition by introducing effective methods, like the fully convolutional network (FCN) [37], residual network (ResNet) and squeeze and excitation (SENet) [20,21,[37][38][39].…”