ReELFA: A Scene Text Recognizer with Encoded Location and Focused Attention

Abstract

LSTM and attention mechanism have been widely used for scene text recognition. However, existing LSTM-based recognizers usually convert 2D feature maps into 1D space by flattening or pooling operations, resulting in the neglect of spatial information of text images. Additionally, the attention drift problem, where models fail to align targets at proper feature regions, has a serious impact on the recognition performance of existing models. To tackle the above problems, in this paper, we propose a scene text Recognizer with Encoded Location and Focused Attention, i.e., ReELFA. Our ReELFA utilizes one-hot encoded coordinates to indicate the spatial relationship of pixels and character center masks to help focus attention on the right feature areas. Experiments conducted on benchmark datasets IIIT5K, SVT, CUTE and IC15 demonstrate that the proposed method achieves comparable performance on the regular, lowresolution and noisy text images, and outperforms state-of-the-art approaches on the more challenging curved text images.

Text Recognition in the Wild: A Survey

“…Motivated by the successes of [180], [181], [68], Su et al [131] used the histogram of oriented gradients (HOG) feature [182] in their STR system to construct sequential features of word images. Later, CNNs [138], [140], [141], [143], [72] have been widely used for feature representation stage, such as the VGGNet [183], [116], [178], [133], [71], [137], [162]. For more powerful feature representation, some complex neural networks were applied in STR algorithms, such as ResNet [184] [135], [145], [148], [73], [151], [166], [152], [154], [48], [155], [157], [158], [161], [163], [164] and DenseNet [185], [150], [160].…”

“…Multiple bidirectional long short term memory (BiLSTM) model was introduced in [188] and widely used in [131], [116], [178], [135], [136], [139], [140], [141], [148], [150], [73], [72], [151], [154], [48], [155], [157], [158], [162], [166] as the sequence modeling module because of its ability to capture long-range dependencies. Litman et al [169] added intermediate supervisions along the network layers and successfully trained a deeper BiLSTM model to improve the encoding of contextual dependencies.…”

“…The attention drift phenomenon means that attention models cannot accurately associate each feature vector with the corresponding target region in the input image. Some researchers added extra information to solve this problem by focusing the deviated attention back onto the target areas, such as localization supervision [71] and encoded coordinates [162]. Others [157], [159], [163] increased the alignment precision of attention in a cascade way.…”