A deep convolutional neural network model for hand gesture recognition in 2D near-infrared images

Can, Celal; Kaya, Yasin; Kılıç, Fatih

doi:10.1088/2057-1976/ac0d91

Cited by 15 publications

(6 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ISLR shares a lot of features with action recognition, and consequently there are several works using CNNs for feature extraction and classification [ 32 , 33 , 34 , 35 ]. Recent work has also relied on employing 3D-CNNs [ 36 , 37 ] to capture spatiotemporal information in an ensemble way.…”

Section: Related Workmentioning

confidence: 99%

Synthetic Corpus Generation for Deep Learning-Based Translation of Spanish Sign Language

Perea-Trigo,

Botella-López,

Martínez-del-Amor

et al. 2024

Sensors

View full text Add to dashboard Cite

Sign language serves as the primary mode of communication for the deaf community. With technological advancements, it is crucial to develop systems capable of enhancing communication between deaf and hearing individuals. This paper reviews recent state-of-the-art methods in sign language recognition, translation, and production. Additionally, we introduce a rule-based system, called ruLSE, for generating synthetic datasets in Spanish Sign Language. To check the usefulness of these datasets, we conduct experiments with two state-of-the-art models based on Transformers, MarianMT and Transformer-STMC. In general, we observe that the former achieves better results (+3.7 points in the BLEU-4 metric) although the latter is up to four times faster. Furthermore, the use of pre-trained word embeddings in Spanish enhances results. The rule-based system demonstrates superior performance and efficiency compared to Transformer models in Sign Language Production tasks. Lastly, we contribute to the state of the art by releasing the generated synthetic dataset in Spanish named synLSE.

show abstract

Section: Related Workmentioning

confidence: 99%

Synthetic Corpus Generation for Deep Learning-Based Translation of Spanish Sign Language

Perea-Trigo,

Botella-López,

Martínez-del-Amor

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…The data is divided 70% for training, and the remainder of the dataset is equally divided between validation and testing. Also, Can, Kaya, et al [19] suggested a CNN model for colored natural ASL pictures and compared their accuracy with five well-known transfer learning models, including VGG16, VGG19, ResNet50, and DenseNet121, to obtain 99.91% superior to his peers.…”

Section: Literature Reviewmentioning

confidence: 99%

“…The result of other state-of-the-art in literature [17] and [13] on collected but not publicly available datasets are also reported in Table VII. The CNN models proposed in [ [46], [19]] achieved higher accuracy on selected images from the MNIST dataset. However, the proposed SL-CNN model is applied to all images in the recent Kaggle dataset, and the two versions of Massey beat all the previously suggested CNN models.…”

Section: Comparison With Other Modelsmentioning

confidence: 99%

An Efficient Convolutional Neural Network Classification Model for Several Sign Language Alphabets

Mahmoud,

Ziedan,

Zamel

2023

IJACSA

View full text Add to dashboard Cite

Although deaf people represent over 5% of the world's population, according to what the World Health Organization stated in May 2022, they suffer from social and economic marginalization. One way to improve the lives of deaf people is to try to make communication between them and others easier. Sign language, the means through which deaf people can communicate with other people, can benefit from modern techniques in machine learning. In this study, several convolutional neural networks (CNN) models are designed to develop an efficient model, in terms of accuracy and computational time, for the classification of different signs. This research presents a methodology for developing an efficient CNN architecture from scratch to classify multiple sign language alphabets, which has numerous advantages over other contemporary CNN models in terms of prediction time and accuracy. This framework analyses the effect of varying CNN hyper-parameters, such as kernel size, number of layers, and number of filters in each layer, and picks the ideal parameters for CNN model construction. In addition, the suggested CNN architecture operates directly on unprocessed data without the need for preprocessing to generalize it across other datasets. In addition, the capacity of the model to generalize to diverse sign languages is rigorously evaluated using three distinct sign language alphabets and five datasets, namely, Arabic (ArSL), two American English (ASL), Korean (KSL), and the combination of Arabic and American datasets. The proposed CNN architecture (SL-CNN) outperforms state-of-the-art CNN models and traditional machine learning models achieving an accuracy of 100%, 98.47%, 100%, and 99.5% for English, Arabic, Korean, and combined Arabic-English alphabets, respectively. The prediction or inference time of the model is about three milliseconds on average, making it suitable for real-time applications. So, in the future, it is easy to turn this model into a mobile application.

show abstract

“…The HGR was provided as an outcome from the text format. Can et al (2021) proposed a DL-CNN approach which categorizes hand gestures efficiently in the investigation of near-infrared and color natural images. This paper presents a novel deep learning (DL) technique dependent upon CNN for recognizing hand gestures, enhancing the rate of recognition, testing, and training time.…”

Section: Related Studiesmentioning

confidence: 99%

Automated Gesture Recognition Using African Vulture Optimization with Deep Learning for Visually Impaired People on Sensory Modality Data

Maashi¹,

Al-Hagery²,

Rizwanullah³

et al. 2023

Journal of Disability Research

View full text Add to dashboard Cite

Gesture recognition for visually impaired persons (VIPs) is a useful technology for enhancing their communications and increasing accessibility. It is vital to understand the specific needs and challenges faced by VIPs when planning a gesture recognition model. But, typical gesture recognition methods frequently depend on the visual input (for instance, cameras); it can be vital to discover other sensory modalities for input. The deep learning (DL)-based gesture recognition method is effective for the interaction of VIPs with their devices. It offers a further intuitive and natural way of relating with technology, creating it more available for everybody. Therefore, this study presents an African Vulture Optimization with Deep Learning-based Gesture Recognition for Visually Impaired People on Sensory Modality Data (AVODL-GRSMD) technique. The AVODL-GRSMD technique mainly focuses on the utilization of the DL model with hyperparameter tuning strategy for a productive and accurate gesture detection and classification process. The AVODL-GRSMD technique utilizes the primary data preprocessing stage to normalize the input sensor data. The AVODL-GRSMD technique uses a multi-head attention-based bidirectional gated recurrent unit (MHA-BGRU) method for accurate gesture recognition. Finally, the hyperparameter optimization of the MHA-BGRU method can be performed by the use of African Vulture Optimization with Deep Learning (AVO) approach. A series of simulation analyses were performed to demonstrate the superior performance of the AVODL-GRSMD technique. The experimental values demonstrate the better recognition rate of the AVODL-GRSMD technique compared to that of the state-of-the-art models.

show abstract

A deep convolutional neural network model for hand gesture recognition in 2D near-infrared images

Cited by 15 publications

References 40 publications

Synthetic Corpus Generation for Deep Learning-Based Translation of Spanish Sign Language

Synthetic Corpus Generation for Deep Learning-Based Translation of Spanish Sign Language

An Efficient Convolutional Neural Network Classification Model for Several Sign Language Alphabets

Automated Gesture Recognition Using African Vulture Optimization with Deep Learning for Visually Impaired People on Sensory Modality Data

Contact Info

Product

Resources

About