Sign language recognition with multi feature fusion and ANN classifier

Abstract

Extracting and recognizing complex human movements such as sign language gestures from video sequences is a challenging task. In this paper this kind of a difficult problem is approached with Indian sign language (ISL) videos.A new segmentation algorithm is developed by fusion of features from discrete wavelet transform (DWT) and local binary pattern (LBP). A 2D point cloud is formed from fused features, which represent the local hand shapes in consecutive video frames. We validate the proposed feature extraction model with state of the art features such as HOG, SIFT and SURF for each sign video on the same ANN classifier. We found that the Haar-LBP fused features represent sign video data in better manner compared to HOG, SIFT and SURF. This is due to the combination of global and local features in our proposed feature matrix. The extracted features input the artificial neural network (ANN) classifier with labels forming the corresponding words. The proposed ANN classifier is tested against state of the art classifiers such as Adaboost, support vector machine (SVM) and other ANN methods on different features extracted from the ISL dataset.The classifiers were tested for accuracy and correctness in identifying the signs. The ANN classifier that produced a recognition rate of 92.79% was obtained with maximum training instances, which was far greater than the existing works on sign language with other features and ANN classifier on our ISL dataset.

Arabic and American Sign Languages Alphabet Recognition by Convolutional Neural Network

“…In the next step, relevant features are extracted from onedimensional signals (a time series from a sensor) or two-dimensional images. Fast Fourier transform [10,11,13], statistical features [8,14], or wavelet transform [15,16] are common methods to extract the useful features from the sensor signals or images. In case of images, Scale-invariant feature transform (SIFT) [17], Histograms of Oriented Gradients (HOG) features [18,19], or speeded up robust features (SURF) [20,21] are common methods to extract features.…”

A Sign Language Recognition Method Relying On Convolutional Recurrent Neural Network

“…Vision-based continuous SLR methods aim to extract spatial and temporal features from continuous image inputs and utilize these features for recognition [17]. Convolutional recurrent neural has a strong ability to extract the spatial features of sign language images [18]. But convolutional recurrent neural cannot exploit the temporal features in consecutive image sequences.…”

Similarity Measurement and Classification of English Characters Based on Language Features

“…e accuracy and correctness of the recognition mark are tested. rough the largest training example, an artificial neural network classifier with a recognition rate of 92.79% is obtained, which is much higher than the existing artificial neural network classifiers on sign language and ISL data sets with other features [3]. is paper mainly studies the temporal retrieval of activities in videos through sentence queries.…”