Text Prior Guided Scene Text Image Super-resolution

Abstract

Scene text image super-resolution (STISR) aims to improve the resolution and visual quality of lowresolution (LR) scene text images, and consequently boost the performance of text recognition. However, most of existing STISR methods regard text images as natural scene images, ignoring the categorical information of text. In this paper, we make an inspiring attempt to embed categorical text prior into STISR model training. Specifically, we adopt the character probability sequence as the text prior, which can be obtained conveniently from a text recognition model. The text prior provides categorical guidance to recover high-resolution (HR) text images. On the other hand, the reconstructed HR image can refine the text prior in return. Finally, we present a multi-stage text prior guided super-resolution (TPGSR) framework for STISR. Our experiments on the benchmark TextZoom dataset show that TPGSR can not only effectively improve the visual quality of scene text images, but also significantly improve the text recognition accuracy over existing STISR methods. Our model trained on TextZoom also demonstrates certain generalization capability to the LR images in other datasets.

Text Gestalt: Stroke-Aware Scene Text Image Super-resolution

“…STT (Chen, Li, and Xue 2021a) contains two text-focused modules including a position-aware module and a content-aware module providing text priors. TPGSR (Ma, Guo, and Zhang 2021) combines text priors in the encoder and employs an iterative manner to enhance low-resolution images. However, these methods usually view characters as the smallest units without considering the more fine-grained details like strokes.…”

“…The recently proposed Scene Text Telescope (STT) (Chen, Li, and Xue 2021a) introduces text priors into the model by proposing a position-aware module and a content-aware module. The concurrent work TPGSR (Ma, Guo, and Zhang 2021) incorporates text-specific semantic features to each block in the backbone and exerts an iterative way to enhance text images. Through observations, the text priors used in these works usually regard character as the smallest unit of text lines, whereas ignoring the significance of more detailed internal structures.…”

Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study

“…The text images are not always of good quality for recognition (e.g., captured with blurring, digital compression, and low-resolution appearance). Although text image recovery has been studied in the last decade, it remains an issue since the performance of current state-of-the-art methods [10,12,48,72,99] are still far from satisfactory and practical use. Besides, the existing methods mainly consider the English text images and seldom take Chinese text images into account.…”

Text Gestalt: Stroke-Aware Scene Text Image Super-Resolution

“…STT (Chen, Li, and Xue 2021a) contains two text-focused modules including a position-aware module and a content-aware module providing text priors. TPGSR (Ma, Guo, and Zhang 2021) combines text priors in the encoder and employs an iterative manner to enhance low-resolution images. However, these methods usually view characters as the smallest units without considering the more fine-grained details like strokes.…”

“…The recently proposed Scene Text Telescope (STT) (Chen, Li, and Xue 2021a) introduces text priors into the model by proposing a position-aware module and a content-aware module. The concurrent work TPGSR (Ma, Guo, and Zhang 2021) incorporates text-specific semantic features to each block in the backbone and exerts an iterative way to enhance text images. Through observations, the text priors used in these works usually regard character as the smallest unit of text lines, whereas ignoring the significance of more detailed internal structures.…”