FOTS: Fast Oriented Text Spotting with a Unified Network

Abstract

Incidental scene text spotting is considered one of the most difficult and valuable challenges in the document analysis community. Most existing methods treat text detection and recognition as separate tasks. In this work, we propose a unified end-to-end trainable Fast Oriented Text Spotting (FOTS) network for simultaneous detection and recognition, sharing computation and visual information among the two complementary tasks. Specially, RoIRotate is introduced to share convolutional features between detection and recognition. Benefiting from convolution sharing strategy, our FOTS has little computation overhead compared to baseline text detection network, and the joint training method learns more generic features to make our method perform better than these two-stage methods. Experiments on ICDAR 2015, ICDAR 2017 MLT, and ICDAR 2013 datasets demonstrate that the proposed method outperforms state-of-the-art methods significantly, which further allows us to develop the first real-time oriented text spotting system which surpasses all previous state-of-theart results by more than 5% on ICDAR 2015 text spotting task while keeping 22.6 fps.

Convolutional Character Networks

“…In this work, we present Convolotional Character Networks (referred as CharNet) for joint text detection and recognition, by leveraging character as basic unit. Moreover, for the first time, we provide an one-stage CNN model for the joint tasks, with significant performance improvements over the state-of-the-art results achieved by a more complex two-stage framework, such as [12], [25] and [24]. The proposed CharNet implements direct character detection and recognition, jointly with text instance (e.g., word) detection.…”

“…First, learning the two tasks independently would result in a sub-optimization problem, making it difficult to fully explore the potential of text nature. For example, text detection and recognition can work collaboratively by providing strong context and complementary information to each other, which is critical to improving the performance, as substantiated by recent work [12,24]. Second, it often requires to implement multiple sequential steps, resulting in a relatively complicated system, where the performance of text recognition is heavily relied on text detection results.…”

“…To overcome the limitations of RoI cropping and pooling for two-stage framework, He et al [12] proposed a text-alignment layer to precisely compute the convolutional features for a text instance of arbitrary orientation, which boosted the performance. In [24], multiple affinity transformations were applied to the convolutional features for enhancing text information in the RoI regions. However, these methods failed to work on curved text.…”

“…These technical improvements result in a simple, compact, yet powerful onestage model that works reliably on multi-orientation and curved text. We evaluate CharNet on three standard benchmarks, where it consistently outperforms the state-of-theart approaches [25,24] by a large margin, e.g., with improvements of 65.33%→71.08% (with generic lexicon) on ICDAR 2015, and 54.0%→69.23% on Total-Text, on endto-end text recognition. Code is available at: https:// github.com/MalongTech/research-charnet.…”

Efficient Scene Text Detection with Textual Attention Tower

“…The detection branches in our proposed method are denoted as DET. Inspired by [1,9], we use rotated box (RBOX) to describe text regions. Thus the DET branch is simply 1 × 1 convolutions to map final feature to detections.…”