Static hand gesture recognition of Arabic sign language by using deep CNNs

Abstract

An Arabic sign language recognition using two concatenated deep convolution neural network models DenseNet121 & VGG16 is presented. The pre-trained models are fed with images, and then the system can automatically recognize the Arabic sign language. To evaluate the performance of concatenated two models in the Arabic sign language recognition, the red-green-blue (RGB) images for various static signs are collected in a dataset. The dataset comprises 220,000 images for 44 categories: 32 letters, 11 numbers (0:10), and 1 for none. For each of the static signs, there are 5000 images collected from different volunteers. The pre-trained models were used and trained on prepared Arabic sign language data. These models were used after some modification. Also, an attempt has been made to adopt two models from the previously trained models, where they are trained in parallel deep feature extractions. Then they are combined and prepared for the classification stage. The results demonstrate the comparison between the performance of the single model and multi-model. It appears that most of the multi-model is better in feature extraction and classification than the single models. And also show that when depending on the total number of incorrect recognize sign image in training, validation and testing dataset, the best convolutional neural networks (CNN) model in feature extraction and classification Arabic sign language is the DenseNet121 for a single model using and DenseNet121 & VGG16 for multi-model using.

Integrated Mediapipe with a CNN Model for Arabic Sign Language Recognition

“…In [55], the authors evaluated classifcation algorithms used in recognition, such as traditional machine learning and deep learning, and discussed previous work on diferentiating between static alphabetic and dynamic sign languages for Arabic and non-Arabic sign languages. In [56], the authors developed a fully automated method for recognizing 28 Arabic signs for letters and numbers.…”

Arabic Sign Language Gesture Classification Using Deer Hunting Optimization with Machine Learning Model

“…Then, agent position is altered until it reaches an effectual location (Y * , Z * ) and changes the location of X and L. Location updating is functioned by the Eq. (10) where p < 1, indicates that an individual is allowable only to move in random way notwithstanding of angle location. Therefore, Eqs.…”

“…The vision-related techniques largely aim at the captured gestures image and receive the primary feature for identifying them. This technique was implied in several tasks, which include semantic segmentation, superresolution, multimedia systems, and emotion recognition and image classification [10].…”

Object detection on robosoccer environment using convolution neural network

“…First object detection is employed to ROBOSOCCER model for detecting the ball/nao and its position in the first frame. Then the next frame is given to the model and the new position is got [36]. Now with the old measurement, the new measurement is updated.…”