American Sign Language Recognition Based on MobileNetV2

Lum, Kin Yun; Goh, Yeh Huann; Lee, Yi Bin

doi:10.25046/aj050657

Cited by 5 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From Table 4, it is clear that the proposed method, whether using a single model or a multi-model, is better than the models presented in the previous studies referred to in the table. [30] Transfer learning using MobileNetV2 on 29 classes 98.67% Sinha et al, 2019 [31] Custom CNN model with fully connected layer on 29 classes 96.03% Kadhim et al, 2020 [32] Transfer learning using VGG1 on 28 classes 98.65% Paul et al 2020 [33] Custom CNN model with fully connected layer on 24 classes 99.02% Mahmud et al, 2018 [34] HOG feature extraction & KNN classifier on 26 classes 94.23% Prasad 2018 [35] Image magnitude gradient for feature extraction on 24 classes 95.40% Phong &Ribeiro 2019 [36] Transfer learning on multiple architecture, etc on 29 classes 99.00% Ashiquzzaman et al, 2020 [37]…”

Section: Resultsmentioning

confidence: 99%

Static hand gesture recognition of Arabic sign language by using deep CNNs

Ismail

Dawwd

Ali

2021

IJEECS

View full text Add to dashboard Cite

An Arabic sign language recognition using two concatenated deep convolution neural network models DenseNet121 & VGG16 is presented. The pre-trained models are fed with images, and then the system can automatically recognize the Arabic sign language. To evaluate the performance of concatenated two models in the Arabic sign language recognition, the red-green-blue (RGB) images for various static signs are collected in a dataset. The dataset comprises 220,000 images for 44 categories: 32 letters, 11 numbers (0:10), and 1 for none. For each of the static signs, there are 5000 images collected from different volunteers. The pre-trained models were used and trained on prepared Arabic sign language data. These models were used after some modification. Also, an attempt has been made to adopt two models from the previously trained models, where they are trained in parallel deep feature extractions. Then they are combined and prepared for the classification stage. The results demonstrate the comparison between the performance of the single model and multi-model. It appears that most of the multi-model is better in feature extraction and classification than the single models. And also show that when depending on the total number of incorrect recognize sign image in training, validation and testing dataset, the best convolutional neural networks (CNN) model in feature extraction and classification Arabic sign language is the DenseNet121 for a single model using and DenseNet121 & VGG16 for multi-model using.

show abstract

Section: Resultsmentioning

confidence: 99%

Static hand gesture recognition of Arabic sign language by using deep CNNs

Ismail

Dawwd

Ali

2021

IJEECS

View full text Add to dashboard Cite

show abstract

“…The model was trained by Squeezenet CNN architecture to make it lighter and capable of running on mobile devices. Another work is proposed by Kin Yun Lum et al [11]. They proposed a CNN model for predicting ASL with a model based on MobileNetV2 transfer learning.…”

Section: Related Workmentioning

confidence: 99%

Azerbaijani sign language recognition using machine learning approach

Aliyev

Almisreb

Turaev

2022

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Sign Language recognition is one of the essential and focal areas for researchers in terms of improving the integration of speech and hearing-impaired people into common society. The main idea is to detect the hand gestures of impaired people and convert them to understandable formats, such as text by leveraging advanced approaches. In this paper, we present our contribution to the improvement of Azerbaijani Sign Language (AzSL). We worked on AzSL Alphabet static signs real-time recognition. The method applied in this work is Object Classification and Recognition by leveraging pre-trained lightweight Convolutional Neural Networks models. At first, a dataset containing near to 1000 images has been collected, then interesting objects on images have been labeled with bounding boxing option. To build, train, evaluate and deploy the relevant model, TensorFlow Object Detection API with Python has been employed. MobileNet v2 pre-trained model has been leveraged for this task. In the trial experiment with four sign classes (A, B, C, E) and 5000 step numbers 15.2% training loss and 83% evaluation mean average precision results have been obtained. In the next step of model deployment experiments with all 24 static signs of AzSL, 49700 and 27700 steps (180 and 100 epochs, respectively) 6.4% and 18.2% training losses, 66.5% and 71.6% mAP outcomes gained, respectively.

show abstract

“…A deep learning technique for processing data that is not too much is using transfer learning, which means that the model has been previously trained with other data [14] [15]. One of the transfer learning techniques from the Keras library is MobileNetV2 [16] and VGG16 [17].…”

Section: Introductionmentioning

confidence: 99%