Synthetic Data for Text Localisation in Natural Images

Abstract

In this paper we introduce a new method for text detection in natural images. The method comprises two contributions: First, a fast and scalable engine to generate synthetic images of text in clutter. This engine overlays synthetic text to existing background images in a natural way, accounting for the local 3D scene geometry. Second, we use the synthetic images to train a Fully-Convolutional Regression Network (FCRN) which efficiently performs text detection and bounding-box regression at all locations and multiple scales in an image. We discuss the relation of FCRN to the recently-introduced YOLO detector, as well as other end-toend object detection systems based on deep learning. The resulting detection network significantly out performs current methods for text detection in natural images, achieving an F-measure of 84.2% on the standard ICDAR 2013 benchmark. Furthermore, it can process 15 images per second on a GPU.

Detecting Oriented Text in Natural Images by Linking Segments

“…In a sense, text detection can be seen as object detection applied to text, where words/characters/text lines are taken as the detection targets. Owing to this, a new trend has emerged recently that state-of-the-art text detection methods [9,6,22,30] are heavily based on the advanced general object detection or segmentation techniques, e.g. [4,5,15].…”

Robust and Accurate Text Stroke Segmentation

“…The FCN is trained on a new data set with 100K synthetically gener- Figure 8. The left column is original images, middle column is our result with text substituted and last column contains samples from dataset generated in [8]. Clearly our synthetic data is more realistic.…”

“…Accurate text stroke segmentation is an important component of text substitution. A typical computational pipeline for text substitution would follow these steps: (1) segment original text strokes; (2) remove text stroke content, substituting with background color or texture; (3) superimpose new text, possibly warped according to the surface orientation [8]. In Fig.8 we show examples of text substitution based on our text stroke segmentation algorithm.…”

“…In [8], the author proposed a method to find "plain" surface on natural images to render text. However, the data set generated by [8] are not fully realistic, in that the distribution of background textures and context information on which text is superimposed may not match the distribution of real world scenario (see Fig.8 last column). By substituting text in regular scenes, we are able to generate new synthetic images (thus increasing the size of training data) while preserving the "natural" background.…”

“…This is the second original contribution of our work. Note that another synthetic data set was created in prior work to facilitate training of text spotters in natural images [8]. However, this prior data set did not provide pixel-level labels, and thus could not be used for our purpose.…”