Dense Chained Attention Network for Scene Text Recognition

FACLSTM: ConvLSTM with focused attention for scene text recognition

“…Comparison with Methods based on the Traditional FC-LSTM: As previously introduced, traditional FC-LSTM is widely used in existing recognizers. Among the methods listed in Table 1, RARE [8], AON [6] and FAN [5] combined FC-LSTM with the attention mechanism in the fully connected way when performing sequential transcription, while CRNN [7], R 2 AM [17], Gao's model [4] and SqueezedText [20] utilized FC-LSTM for frame-level prediction, sequential feature encoding or other purposes. As shown in Table 1, our proposed FACLSTM outperforms these FC-LSTM-based methods by large margins on both regular text dataset IIIT5K (90.5% vs 87.4%) and curved text dataset CUTE (83.33% and 76.8%) when no lexicon is used.…”

“…Methods based on LSTM: LSTM is widely used in the existing state-of-the-art recognizers for three purposes, i.e., producing frame-level predictions required by the subsequent sequential transcription module [4,7], encoding sequential features with considering historical information [8,16], and directly generating sequential predictions when cooperating with the attention mechanism [5,6,13,16,17]. For example, CRNN proposed by Shi et al [7] was composed of three parts, i.e., the convolution module used to extract features from input images, a bi-LSTM layer built to make predictions for individual frames, and a CTC-based sequential transcription component utilized to infer sequential outputs from frame-level predictions.…”

“…In other recognizers [4,10,12], the feature maps F and maps A generated for other purposes are always combined with the element-wise multiplication ⊗ in the way of F out = F ⊗ (1 + A). However, in our experiments we find that directly concatenating feature maps F and character center masks M achieves better performance, which means the subsequent attention-based module prefers to learn patterns from F and M directly, rather than from their fused results.…”

“…However, the LSTM used in these recognizers is the fully-connected-LSTM (FC-LSTM) that only takes stream signals like sentences or audio as inputs and connects them in a fully connected way, while scene text recognition generates sequential outputs from 2-D images. To adapt FC-LSTM to scene text recognition, the most straightforward way is pooling 2-D feature maps to a height of one or flattening them into 1-D sequential feature vectors [4,5,6,7,8], as shown in Fig 2(a). Unfortunately, such operations could severely disrupt the valuable spatial correlation relationships among pixels, which is essential to computer vision tasks, especially to scene text recognition, where the structures of strokes are the key factors to discriminate characters.…”

“…However, in existing models, both FC-LSTM and ConvLSTM are used only for frame-level prediction and are incapable of producing sequential outputs from one single input image unless the Connectionist Temporal Classification (CTC) [4,7,12] or attention mechanism [5,6,8,13] is incorporated. To perform sequential prediction and, meanwhile, provide the model spatial awareness, we further improve ConvL-STM by embedding the attention mechanism into the structure.…”