A New Multi-modal Technique for Bib Number/Text Detection in Natural Images

Abstract

Abstract. The detection and recognition of racing bib number/text, which is printed on paper, cardboard tag, or t-shirt in natural images in marathon, race and sports, is challenging due to person movement, non-rigid surface, distortion by non-illumination, severe occlusions, orientation variations etc. In this paper, we present a multi-modal technique that combines both biometric and textual features to achieve good results for bib number/text detection. We explore face and skin features in a new way for identifying text candidate regions from input natural images. For each text candidate region, we propose to use text detection and recognition methods for detecting and recognizing bib numbers/texts, respectively. To validate the usefulness of the proposed multi-modal technique, we conduct text detection and recognition experiments before text candidate region detection and after text candidate region detection in terms of recall, precision and f-measure. Experimental results show that the proposed multi-modal technique outperforms the existing bib number detection method.

Racing Bib Number Localization Based on Region Convolutional Neural Networks

“…Ami et al [2] focuses on the stroke width transform to find text with related stroke width within an RBN image. Ami et al The multi-modal technique [3], a number of steps to detect the real RBN, improved the accuracy to 64%, employed on 212 images, but there is no time of computation described. Disadvantage of this technique, the color based method is utilized which is very sensitive to lighting conditions.…”

“…Future work will attempt to solve these problems, which will improve the performance by using the CIFAR-100 dataset. 3(a) detect other circular object 3(b) detect other tag( similar shape)…”

“…The detection accuracy of Stroke Width Transform technique is at 45% (f-measure). To improve the accuracy, Roy et al [3] proposed a multi-modal technique. First, face and skin color are used to detect runner's body in natural and marathon images.…”

Racing Bib Number Recognition Method using Deep Learning

“…One of its limitations is when the participant's face is not detected in the image due to the photo's inconsistent angle hence the RBN recognition cannot be done. Inspired by Ami et al following work, Roy et al proposed a method using a multi-modal technique that combines face, skin and text detection to extract text candidate region as participant generally displays skin and then used it as a cue to identify a person body which gives the general position about RBN whereabout within an image [6]. Extracted text candidate region was applied with a text detection technique which utilizes a combination of wavelet and color, and k-mean clustering to detect text line.…”

“…,(5), and(6): predicted bounding box class order from "class.txt" in integer form <x-center>Thex-center coordinate of object bounding box relative to the image resolution <y-center> The y-center coordinate of object bounding center relative to the image resolution <width> The width of the object bounding box relative to the image resolution <height> The height of the object bounding box relative to the image resolution Juli 2020, Page 815-824 ISSN 2614-5278 (media cetak), ISSN 2548-8368 (media online) Available Online at https://ejurnal.stmik-budidarma.ac.id/index.php/mib DOI 10.30865/mib.v4i3.2268 Muhammad Aditya Rayhan, Copyright ©2020, MIB, Page 819 Submitted: 07/06/2020; Accepted: 30/06/2020; Published:…”