Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting

Abstract

Many approaches have recently been proposed to detect irregular scene text and achieved promising results. However, their localization results may not well satisfy the following text recognition part mainly because of two reasons: 1) recognizing arbitrary shaped text is still a challenging task, and 2) prevalent non-trainable pipeline strategies between text detection and text recognition will lead to suboptimal performances. To handle this incompatibility problem, in this paper we propose an end-to-end trainable text spotting approach named Text Perceptron. Concretely, Text Perceptron first employs an efficient segmentation-based text detector that learns the latent text reading order and boundary information. Then a novel Shape Transform Module (abbr. STM) is designed to transform the detected feature regions into regular morphologies without extra parameters. It unites text detection and the following recognition part into a whole framework, and helps the whole network achieve global optimization. Experiments show that our method achieves competitive performance on two standard text benchmarks, i.e., ICDAR 2013 and ICDAR 2015, and also obviously outperforms existing methods on irregular text benchmarks SCUT-CTW1500 and Total-Text.

FREE: A Fast and Robust End-to-End Video Text Spotter

“…A. Spatial-Temporal Video Text Detector 1) Text Detection in Single Frame: Since TP [17] is a more robust text spotter than EAST [16] especially on irregular text detection, and also can be trained end-to-end. We redesign and implement the video text detector inspired by the TP architecture (including a text detection module, a shape transform module and a recognition module), as shown in Figure 2.…”

“…With the rapid development of artificial intelligence techniques [18], [19], [20], [21], great progress has been made in many isolated applications such as causal inference [22], named entities identification [23], question answering [24], scene text spotting [5], [6], [17] and video understanding [25], [26]. However, it is very important to build multiple knowledge representation [27] for understanding the real and complex world.…”

“…Recently, in order to sufficiently exploit the complementarity between detection and recognition, many methods [45], [4], [5], [6], [46], [7], [17], [47], [37], [48], [49], [50] are proposed to spot text in an end-to-end manner, which utilize the recognition information to optimize the localization task.…”

“…In fact, it is a very challenging task to optimize video text spotter end-to-end when taking multiple functional modules (text detection, text tracking and text recognition) into consideration, especially compared to the traditional four-staged pipeline strategy. Therefore, in this paper we develop an endto-end trainable video text spotter with only two trainable modules: the video text detector and the text recommender, similar to the end-to-end text spotting methods [6], [17], [45], [47], [48], [49] in single images.…”

“…Declaration of major extensions compared to the conference version [8]: (1) We achieve the video text spotting in an end-to-end trainable manner instead of the two-staged form in its conference version. To achieve this, we replace EAST [16] with an end-to-end trainable text spotting framework Text Perceptron [17] (abbr. TP), in which the original recognition module in TP is replaced with our text recommender submodule.…”