Focusing Attention: Towards Accurate Text Recognition in Natural Images

Abstract

Scene text recognition has been a hot research topic in computer vision due to its various applications. The state of the art is the attention-based encoder-decoder framework that learns the mapping between input images and output sequences in a purely data-driven way. However, we observe that existing attention-based methods perform poorly on complicated and/or low-quality images. One major reason is that existing methods cannot get accurate alignments between feature areas and targets for such images. We call this phenomenon "attention drift". To tackle this problem, in this paper we propose the FAN (the abbreviation of Focusing Attention Network) method that employs a focusing attention mechanism to automatically draw back the drifted attention. FAN consists of two major components: an attention network (AN) that is responsible for recognizing character targets as in the existing methods, and a focusing network (FN) that is responsible for adjusting attention by evaluating whether AN pays attention properly on the target areas in the images. Furthermore, different from the existing methods, we adopt a ResNet-based network to enrich deep representations of scene text images. Extensive experiments on various benchmarks, including the IIIT5k, SVT and ICDAR datasets, show that the FAN method substantially outperforms the existing methods.Comment: Revise the description of IC15 datasets (1811 samples

Convolutional Character Networks

“…This inevitably makes the models more complicated and difficult to train by requiring a significantly longer training time with a large amount of training samples. Recent work, such as [33,4,12], had shown that the performance of RNN-based methods can be improved considerably by introducing char-level attention mechanism which is able to encode strong character information implicitly or explicitly. This enables the models to have the ability to identify characters more accurately, and essentially adds additional constraints to the models which in turn reduce the search space, leading to performance boost.…”

Scene Text Recognition with Temporal Convolutional Encoder

“…Among them, the first group contains several well-known recognition networks, including CRNN [1] and GRCNN [5]. We then compare ours with previous attention aware approaches such as FAN [8], FCN [14], and Baek et al [16].…”

“…The attention mechanism is incorporated into the decoder. For example, Lee et al [7] proposed to use an attention-based decoder for text-output prediction, while Cheng et al [8] presented the Focusing Attention Network (FAN) to tackle attention drift problem in order to improve the performance of regular text recognition. Besides, some previous work also exploited to handle the irregular scene text images at the beginning of the encoder.…”

Adaptive embedding gate for attention-based scene text recognition

“…Generally, in the encoding stage, the convolutional neural networks (CNN) are used to extract features from the input image, whereas in the decoding stage, the encoded feature vectors are transcribed into target strings by exploiting the recurrent neural network (RNN) [12], [13], (c) The recurrent decoding process of attention-based decoder with AEG i a n a connectionist temporal classification (CTC) [14] or attention mechanism [15]. In particular, the attention-based approaches [4], [11], [16], [17], [18] often achieve better performance owing to the focus on informative areas.…”