Chinese Fingerspelling Recognition via Hu Moment Invariant and RBF Support Vector Machine

Aim) Fingerspelling is a method of spelling words via hand movements. This study aims to propose a novel fingerspelling recognition system. (Method) We use 1320 fingerspelling images in our dataset. Our method is based on the convolutional neural network (CNN) model. We propose a 12-layer CNN as the backbone. Particularly, stochastic pooling (SP) is used to help solve the problems caused by max pooling or average pooling. In addition, an improved 20-way data augmentation method is proposed to circumvent overfitting. Our method is dubbed CNNSP.(Results) The results show that our CNNSP method achieved a micro-averaged F1 (MAF) score of 90.04±0.82%. In contrast, the MAFs of l2-pooling, average pooling, and max pooling are 86.21±1.12%, 87.54±1.39%, and 89.07±0.78%, respectively. (Conclusion) Our CNNSP attains better results than eight state-of-the-art fingerspelling recognition methods. Besides, the SP is better than l2-pooling, average pooling, and max pooling.

show abstract

“…HMM [5], SVM [6], HMI-kSVM [7], POMGSVM [8], WE [9], 9L-CNN [10], and FSVM [11]. The results are shown in Table 6.…”

Section: Comparison To State-of-the-art Methodsmentioning

confidence: 99%

“…Lee, Yeh and Hsiao (2016) [6] used a support vector machine (SVM) to recognize Taiwanese sign language. Gao (2020) [7] combined Hu moment invariants (HMIs) with a kernel support vector machine (kSVM). Their method is abbreviated as HMI-kSVM.…”

Section: Introductionmentioning

confidence: 99%

Fingerspelling Recognition by 12-Layer CNN with Stochastic Pooling

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…For instance, Scale Invariant Feature Transform (SIFT) [27][28][29] can always be employed to extract the features of the sign language image as the sign language visual vocabulary in the image. In addition, Hu moment invariant (HMI) [30], Fourier descriptors (FD) [31,32], Speeded Up Robust Features (SURF) [33], and Latent Dirichlet Allocation (LDA) [34], etc., also appear frequently in some papers.…”

Section: Other Feature Extraction Approachesmentioning

confidence: 99%

A Survey on Chinese Sign Language Recognition: From Traditional Methods to Artificial Intelligence

Jiang,

Zhang,

Lei

et al. 2024

CMES

View full text Add to dashboard Cite

Research on Chinese Sign Language (CSL) provides convenience and support for individuals with hearing impairments to communicate and integrate into society. This article reviews the relevant literature on Chinese Sign Language Recognition (CSLR) in the past 20 years. Hidden Markov Models (HMM), Support Vector Machines (SVM), and Dynamic Time Warping (DTW) were found to be the most commonly employed technologies among traditional identification methods. Benefiting from the rapid development of computer vision and artificial intelligence technology, Convolutional Neural Networks (CNN), 3D-CNN, YOLO, Capsule Network (CapsNet) and various deep neural networks have sprung up. Deep Neural Networks (DNNs) and their derived models are integral to modern artificial intelligence recognition methods. In addition, technologies that were widely used in the early days have also been integrated and applied to specific hybrid models and customized identification methods. Sign language data collection includes acquiring data from data gloves, data sensors (such as Kinect, Leap Motion, etc.), and high-definition photography. Meanwhile, facial expression recognition, complex background processing, and 3D sign language recognition have also attracted research interests among scholars. Due to the uniqueness and complexity of Chinese sign language, accuracy, robustness, real-time performance, and user independence are significant challenges for future sign language recognition research. Additionally, suitable datasets and evaluation criteria are also worth pursuing.

show abstract