Recurrent Calibration Network for Irregular Text Recognition

Abstract

Scene text recognition has received increased attention in the research community. Text in the wild often possesses irregular arrangements, typically including perspective text, curved text, oriented text. Most existing methods are hard to work well for irregular text, especially for severely distorted text. In this paper, we propose a novel Recurrent Calibration Network (RCN) for irregular scene text recognition. The RCN progressively calibrates the irregular text to boost the recognition performance. By decomposing the calibration process into multiple steps, the irregular text can be calibrated to normal one step by step. Besides, in order to avoid the accumulation of lost information caused by inaccurate transformation, we further design a fiducial-point refinement structure to keep the integrity of text during the recurrent process. Instead of the calibrated images, the coordinates of fiducial points are tracked and refined, which implicitly models the transformation information. Based on the refined fiducial points, we estimate the transformation parameters and sample from the original image at each step. In this way, the original character information is preserved until the final transformation. Such designs lead to optimal calibration results to boost the performance of succeeding recognition. Extensive experiments on challenging datasets demonstrate the superiority of our method, especially on irregular benchmarks.

Revisiting Classification Perspective on Scene Text Recognition

“…ASTER [37] adds a spatial transformer before the scene text recognition model to eliminate the negative effects of perspective distortion and distribution curvature which boosts performance on irregular texts. Since the proposal of spatial transformer in [28], spatial transformer has been a default module on scene text recognition [36,12,37,46,48,16,27,33].…”

Character Region Attention for Text Spotting

“…To help recognition networks handle irregular texts, some researches [36,28,37] utilize spatial transformer network (STN) [18]. Also, the papers [11,46] further extend the use of STN by iterative executing the rectification method. These studies show that running STN recursively helps recognizer extract useful features in extremely curved texts.…”

“…The detectors [32,31,2] attempt to capture the geometric attributes of curved texts Corresponding author. by applying complicated post-processing techniques, and the recognizers apply multi-directional encoding [6] or take rectification modules [37,46,11] to enhance the accuracy of the recognizer on curved texts.…”

On Recognizing Texts of Arbitrary Shapes with 2D Self-Attention

“…By design, such models fail to address curved or rotated text. To overcome this issue, spatial transformation networks (STN) have been applied to align text image into a canonical shape (horizontal alignment and uniform character widths and heights) (Shi et al 2016;Liu, Chen, and Wong 2018;Liu et al 2016;Gao et al 2018). STN does handle non-canonical text shapes to some degree, but is limited by the hand-crafted design of transformation space and the loss in fine details due to image interpolation.…”

“…There are largely two lines of research: (1) input rectification and (2) usage of 2D feature maps. Input rectification (Shi et al 2016;Liu, Chen, and Wong 2018;Liu et al 2016;Gao et al 2018) uses spatial transformer networks (STN, (Jaderberg et al 2015)) to normalize text images into canonical shapes: horizontally aligned characters of uniform heights and widths. These methods, however, suffer from the limitation that the possible family of transformations have to be specified beforehand.…”