A Large Chinese Text Dataset in the Wild

Abstract

We introduce Chinese Text in the Wild, a very large dataset of Chinese text in street view images. While optical character recognition (OCR) in document images is well studied and many commercial tools are available, detection and recognition of text in natural images is still a challenging problem, especially for more complicated character sets such as Chinese text. Lack of training data has always been a problem, especially for deep learning methods which require massive training data.In this paper we provide details of a newly created dataset of Chinese text with about 1 million Chinese characters annotated by experts in over 30 thousand street view images. This is a challenging dataset with good diversity. It contains planar text, raised text, text in cities, text in rural areas, text under poor illumination, distant text, partially occluded text, etc. For each character in the dataset, the annotation includes its underlying character, its bounding box, and 6 attributes. The attributes indicate whether it has complex background, whether it is raised, whether it is handwritten or printed, etc. The large size and diversity of this dataset make it suitable for training robust neural networks for various tasks, particularly detection and recognition. We give baseline results using several state-ofthe-art networks, including AlexNet, OverFeat, Google Inception and ResNet for character recognition, and YOLOv2 for character detection in images. Overall Google Inception has the best performance on recognition with 80.5% top-1 accuracy, while YOLOv2 achieves an mAP of 71.0% on detection. Dataset, source code and trained models will all be publicly available on the website 1 . 1 https://ctwdataset.github.io/ Figure 1. High intra-class variance versus low inter-class variance.Each row shows instances of a Chinese character. The first character differs from the second character by a single stroke, and the second character differs from the third character by another stroke. While the three characters are very similar in shape, the instances of the same character have very different appearance, due to color, font, occlusion, and background differences, etc. The most right column shows the corresponding Chinese character.

Chinese Street View Text: Large-Scale Chinese Text Reading With Partially Supervised Learning

“…For Chinese text, Liu et al [26] first introduced a dataset for the online and offline handwritten recognition. For Chinese text in the wild, MSRA-TD500 [42], RCTW-17 [36] and CTW [43] have been released to evaluate the performance of Chinese text reading models. Unlike all the previous datasets which only provide fully annotated images, the proposed C-SVT dataset also introduces a large amount of weakly annotated images with only the text labels in regions-of-interest, which are much easier to collect and have the potential to further improve the performance of text reading models.…”

“…Unlike all the previous datasets which only provide fully annotated images, the proposed C-SVT dataset also introduces a large amount of weakly annotated images with only the text labels in regions-of-interest, which are much easier to collect and have the potential to further improve the performance of text reading models. C-SVT is at least 14 times as large as the previous Chinese benchmarks [36,43], making it the largest dataset for reading Chinese text in the wild.…”

“…Previous end-to-end trainable text reading models [40,14,22,28,29] only utilize images in full annotations provided by the previous benchmarks [36,3,43]. The improvement in performance of these models requires more fully annotated training data, which is extremely expensive and inefficient in annotations.…”

“…Since the category number of Chinese characters in real-world images is much larger than those of Latin languages, the number of training samples of most current datasets is still limited per category and the distribution of characters is relatively unbalanced. Therefore, reading Chinese text in the wild requires more well annotated training samples, however, it is difficult for the existing benchmarks [36] [43] to satisfy the requirements mainly due to the high cost of data collections and location annotations of text regions.…”

ICDAR 2019 Competition on Large-Scale Street View Text with Partial Labeling - RRC-LSVT

“…Recent powerful deep learning models contributed dramatically to the advances of robust text reading problems, including text detection, recognition and end-to-end text spotting. Benefiting from the pioneer work of the existing benchmarks [1], [2], [3], [4], [5], [6], [7], [8], [9], remarkable success has been achieved in text detection and recognition in the wild. Since most of the scene text datasets provide fully annotated ground truth (i.e.…”