GLASS: Global to Local Attention for Scene-Text Spotting

Abstract

In recent years, the dominant paradigm for text spotting is to combine the tasks of text detection and recognition into a single endto-end framework. Under this paradigm, both tasks are accomplished by operating over a shared global feature map extracted from the input image. Among the main challenges that end-to-end approaches face is the performance degradation when recognizing text across scale variations (smaller or larger text), and arbitrary word rotation angles. In this work, we address these challenges by proposing a novel global-to-local attention mechanism for text spotting, termed GLASS , that fuses together global and local features. The global features are extracted from the shared backbone, preserving contextual information from the entire image, while the local features are computed individually on resized, high resolution rotated word crops. The information extracted from the local crops alleviates much of the inherent difficulties with scale and word rotation. We show a performance analysis across scales and angles, highlighting improvement over scale and angle extremities. In addition, we introduce an orientation-aware loss term supervising the detection task, and show its contribution to both detection and recognition performance across all angles. Finally, we show that GLASS is general by incorporating it into other leading text spotting architectures, improving their text spotting performance. Our method achieves state-of-the-art results on multiple benchmarks, including the newly released TextOCR.

Temporally-aware Convolutional Block Attention Module for Video Text Detection

“…As shown in Table 2, we have summarized various text spotting methods. We see that A3S surpasses the recent competitive methods [1,5,14,15] on most of the metrics. In CTW1500, our method achieves 64.4 and 82.3 for "None" and "Full", respectively.…”

“…Similar to CTW1500, we confirm that A3S performs well when unavailable lexicons. The proposed method performs better than GLASS [15], which enhances features before text recognition based on the global information in an input image. It is suggested that not only visual information but also semantic one is effective in scenetext spotting.…”

“…In contrast, our work focuses on improving text recognition on scene-text spotting. Although GLASS [15] proposed an attention mechanism recently to fuse visual global and local information for text recognition accuracy, it only relies on vulnerable visual features. Our method exploits the semantic representations of the text to improve recognition.…”

Towards End-to-End Unified Scene Text Detection and Layout Analysis

“…Text detection and recognition systems [11] and geometric layout analysis techniques [12,13] have long been developed separately as independent tasks. Research on text detection and recognition [14,15,16,17] has mainly focused on the domain of natural images and aimed at single level text spotting (mostly, wordlevel). Conversely, research on geometric layout analysis [12,13,18,19], which is targeted at parsing text paragraphs and forming text clusters, has assumed document images as input and taken OCR results as fixed and given by independent systems.…”