A Survey on Artificial Intelligence in Chinese Sign Language Recognition

Abstract

Chinese Sign Language (CSL) offers the main means of communication for the hearing impaired in China. Sign Language Recognition (SLR) can shorten the distance between the hearing-impaired and healthy people and help them integrate into the society. Therefore, SLR has become the focus of sign language application research. Over the years, the continuous development of new technologies provides a source and motivation for SLR. This paper aims to cover the most recent approaches in Chinese Sign Language Recognition (CSLR). With a thorough review of superior methods from 2000 to 2019 in CSLR researches, various techniques and algorithms such as scale-invariant feature transform, histogram of oriented gradients, wavelet entropy, Hu moment invariant, Fourier descriptor, gray-level co-occurrence matrix, dynamic time warping, principal component analysis, autoencoder, hidden Markov model (HMM), support vector machine (SVM), random forest, skin color modeling method, k-NN, artificial neural network, convolutional neural network (CNN), and transfer learning are discussed in detail, which are based on several major stages, that is, data acquisition, preprocessing, feature extraction, and classification. CSLR was summarized from some aspect as follows: methods of classification and feature extraction, accuracy/performance evaluation, and sample size/datasets. The advantages and limitations of different CSLR approaches were compared. It was found that data acquisition is mainly through Kinect and camera, and the feature extraction focuses on hand's shape and spatiotemporal factors, but ignoring facial expressions. HMM and SVM are used most in the classification. CNN is becoming more and more popular, and a deep neural network-based recognition approach will be the future trend. However, due to the complexity of the contemporary Chinese language, CSLR generally has a lower accuracy than other SLR. It is necessary to establish an appropriate dataset to conduct comparable experiments. The issue of decreasing accuracy as the dataset increases needs to resolve. Overall, our study is hoped to give a comprehensive presentation for those people who are interested in CSLR and SLR and to further contribute to the future research.

Facial emotion recognition via stationary wavelet entropy and Biogeography-based optimization

“…In the future research, we will test their performances. And we will also attempt to use deep learning approaches [49][50][51][52][53] to realize facial emotion recognition.…”

Gingivitis detection by Fractional Fourier Entropy and Particle Swarm Optimization

“…It provides a new automatic detection model for the diagnosis of gingivitis and extends the performance and function of the intelligent gingival diagnosis system. This method provides a new direction for auxiliary dentists to diagnose gingivitis, and has important significance in promoting the diagnostic efficiency of neural network performance testing [40]. In the future, we will optimize the gingivitis detection algorithm to obtain a superior accuracy and a less burdensome method.…”

Tea category classification via 5-layer customized convolutional neural network

“…Meanwhile, we also try to use swarm intelligence optimization algorithms to optimize the values of hyperparameters in the network. Some advanced CNN techniques [38], such as advanced pooling [39] and attention neural network [40], will be tested. , ( ) Output of BN when the input data is .…”