Sign and Human Action Detection Using Deep Learning

Abstract

Human beings usually rely on communication to express their feeling and ideas and to solve disputes among themselves. A major component required for effective communication is language. Language can occur in different forms, including written symbols, gestures, and vocalizations. It is usually essential for all of the communicating parties to be fully conversant with a common language. However, to date this has not been the case between speech-impaired people who use sign language and people who use spoken languages. A number of different studies have pointed out a significant gaps between these two groups which can limit the ease of communication. Therefore, this study aims to develop an efficient deep learning model that can be used to predict British sign language in an attempt to narrow this communication gap between speech-impaired and non-speech-impaired people in the community. Two models were developed in this research, CNN and LSTM, and their performance was evaluated using a multi-class confusion matrix. The CNN model emerged with the highest performance, attaining training and testing accuracies of 98.8% and 97.4%, respectively. In addition, the model achieved average weighted precession and recall of 97% and 96%, respectively. On the other hand, the LSTM model’s performance was quite poor, with the maximum training and testing performance accuracies achieved being 49.4% and 48.7%, respectively. Our research concluded that the CNN model was the best for recognizing and determining British sign language.

Sign Language Recognition Using Artificial Rabbits Optimizer with Siamese Neural Network for Persons with Disabilities

“…In Table 3 and Figure 9, the overall results of the SLR-AROSNN method are compared with recent approaches (Dhulipala et al, 2022). Based on accu y , the SLR-AROSNN of 98.77, 98.62, 95.42, 97.32, and 98.63%, correspondingly.…”

Machine Learning-Based Gesture Recognition for Communication with the Deaf and Dumb

“…We'll connect each neuron to an input volume section. Convolutional layers form CNNs (Rao et al, 2023b), the layer's parameters are a set of learnable kernels with a small receptive field but full input volume (Liu et al, 2023;Dhulipala et al, 2022). In the forward pass, each filter convolved across the width and height of the input volume computes the dot product between its entries and the input and creates a 2-dimensional activation map shown in Figure 2.…”

Recognizing British Sign Language Using Deep Learning: A Contactless and Privacy-Preserving Approach

“…CNN and LSTM employ vision-based data. The CNN model performed the best, achieving training and testing accuracies of 98.8% and 97.4%, whereas the LSTM model performed poorly, achieving training and testing accuracies of 49.4% and 48.8%, respectively [29]. This study presents the use of a four-beam patch antenna as a sensor node to evaluate the pillrolling effect in Parkinson's disease using the S-band sensing technique at 2.4 GHz.…”