A New DCT-FFT Fusion Based Method for Caption and Scene Text Classification in Action Video Images

Abstract

Achieving better recognition rate for text in video action images is challenging due to multi-type texts with unpredictable backgrounds. We propose a new method for the classification of captions (which is edited text) and scene texts (which is part of an image in video images of Yoga, Concert, Teleshopping, Craft, and Recipe classes). The proposed method introduces a new fusion criterion-based on DCT and Fourier coefficients to extract features that represent good clarity and visibility of captions to separate them from scene texts. The variances for coefficients of corresponding pixels of DCT and Fourier images are computed to derive the respective weights. The weights and coefficients are further used to generate a fused image. Furthermore, the proposed method estimates sparsity in Canny edge image of each fused image to derive rules for classifying caption and scene texts. Lastly, the proposed method is evaluated on images of five above-mentioned action image classes to validate the derived rules. Comparative studies with the state-of-the-art methods on the standard databases show that the proposed method outperforms the existing methods in terms of classification. The recognition experiments before and after classification show that the recognition performance rate improves significantly after classification.

A deep action-oriented video image classification system for text detection and recognition

“…However, the methods are not tested on action images without text information. Recently, the method [42] proposes the combination of Discrete Cosine Transform and Fast Fourier Transform for classifying caption and scene texts in action images to improve text recognition results. The method generates a fused image for the input and then the average of sparsity and non-sparsity counts in terms pixel values of zero or non-zeros is computed for classification.…”

“…6 Sample images of successful classification of the proposed modelon our dataset. Original Source: [42] https://doi.org/10.1007/s42452-021-04821-z of classes increases, the complexity of the problem also increases. But if we consider the overall performance in terms of classification rate, the proposed method outperforms the others.…”

“…13 Samples of unsuccessful classification results of the proposed approach on different datasets. Original Source: [42] and [8]…”

“…The images used in the proposed work are originally taken from [8,42]. The authors of this paper and the authors of [8,42] have given consent to use the data for publication.…”

“…Qualitative results of the text detection[9] on STD dataset before and after classification. Original Source:[8,42]…”