Intelligent Dynamic Gesture Recognition Using CNN Empowered by Edit Distance

Vision-based Pakistani Sign Language Recognition Using Bag-of-Words and Support Vector Machines

“…Most of these studies extract specific features and then use machine learning algorithms to classify the SL images. Many different SL have been used in these studies, namely American SL [1][2][3][4][5][6][7][8] , Arabic SL [9][10][11][12] , British SL 13 , Chinese SL 14 , German SL 15,16 , Indian SL 17 , Irish SL 18 , Pakistani SL [19][20][21][22][23][24] , Persian SL 25 , and more in combination such as American & German SL 26 , American & Thai SL 27 and American & Indian SL 28 .…”

“…The literature review was done for Pakistani SL (PSL) to identify the protocols used for the collection of data for static and dynamic PSL alphabets and the methods used for the recognition of PSL alphabets. The protocol used by the researchers of all the included PSL studies used RGB images and single-handed static signs of PSL alphabets except for Saqib, e.t.al, who used dynamic PSL words24 . The studies used various lighting conditions and studies by Kausar, e.t.al19 , and Shah, e.t.al23 , mentioned that the clothing should be separate from the skin colour of the participant.…”

“…Another study by Farman Shah, e.t.al23 , used SURF with SVM but obtained 15.41% accuracy and the final reported accuracy using Histogram of Oriented Gradient (HOG) and SVM was 91.98%, which was also the highest classification accuracy reported, to the best of my knowledge, using static PSL alphabets. Shazia Saqib, e.t.al24 , used dynamic PSL words with CNN with Levenshtein distance to obtain 90.79% accuracy.The highest classification accuracy obtained in this study for static PSL signs, was 97.80% as compared to 96.23% by Nasser H. Dardas, e.t.al7 , who used 1,000 static American SL images with non-uniform lighting, background, scale and rotation, and 15.41% by Farman Shah, e.t.al23 , who used SURF directly with SVM, instead of using the BoW technique. No previously performed study has classified dynamic PSL, to the best of my knowledge, so the classification accuracy of 96.53% for dynamic PSL signs cannot be compared to any PSL study.…”

“…Instead of sensor-based recognition systems, i.e., systems that use Cyber-gloves, leap motion controller, accelerometers or EMG sensors, vision-based SL recognition systems were focused. Specifically, those systems that used images and videos from a single camera of bare hands, instead of those that used multiple cameras or different object tracking technologies, were used for this study.Many systems used a combination of image and video-based datasets as input and used different classifiers, such as, Neural Networks like Convolutional Neural Network (CNN) and Multilayer Perceptron (MLP), Support Vector Machine (SVM), K Nearest Neighbor (KNN), Hidden Markov Model (HMM), etc.Saqib, e.t.al, used 20 dynamic PSL words, with 8,000 videos (6,480 training /1,520 testing) collected from 15 participants, resized the images to 234×234 and converted them to grayscale, and used CNN with Convolution layers and fully connected layers, along functional layers such as max pooling Layers, Rectified Linear Units layer (ReLU layer) and SoftMax activation function to achieve a 90.79% accuracy24 . Shah, e.t.al, classified 36 PSL alphabets, with 6,633 images (4,643 training /1,990 testing) collected from 6 participants using SVM and using K-means clusteringbased segmentation and converting them to grayscale, obtained classification accuracies of 15.41% using Speeded Up Robust Features (SURF), 87.67% using Edge Orientation Histogram (EOH), 45.71% using Local Binary Patterns (LBP), and 89.52% using Histogram of Oriented Gradient (HOG) and the final reported accuracy of 91.98%23 .…”

Gesture Recognition of Somatosensory Interactive Acupoint Massage Based on Image Feature Deep Learning Model

“…The CNN is very suitable for image processing. With the speed improvement of computer hardware, the CNN has been widely adopted for face recognition [24,25], gesture recognition [26,27], human pose recognition [28], license plate recognition [29,30], security, and other fields. In this paper, our CNN for pose recognition in acupoint massage includes four parts: convolutional layer, activation layer, pooling layer, and fully connected layer.…”