American Sign Language alphabet recognition using Convolutional Neural Networks with multiview augmentation and inference fusion

Tao, Wenjin; Leu, Ming-Chuan; Yin, Zhaozheng

doi:10.1016/j.engappai.2018.09.006

Cited by 130 publications

(57 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Tao et al (2018) [22] proposed a Convolutional Neural Network (CNN) along with inference fusion and multiview augmentation. This method uses depth images captured by Microsoft Kinect.…”

mentioning

confidence: 99%

Gesture Recognition of RGB and RGB-D Static Images Using Convolutional Neural Networks

Khari¹,

Garg²,

Crespo³

et al. 2019

IJIMAI

View full text Add to dashboard Cite

In this era, the interaction between Human and Computers has always been a fascinating field. With the rapid development in the field of Computer Vision, gesture based recognition systems have always been an interesting and diverse topic. Though recognizing human gestures in the form of sign language is a very complex and challenging task. Recently various traditional methods were used for performing sign language recognition but achieving high accuracy is still a challenging task. This paper proposes a RGB and RGB-D static gesture recognition method by using a fine-tuned VGG19 model. The fine-tuned VGG19 model uses a feature concatenate layer of RGB and RGB-D images for increasing the accuracy of the neural network. Finally, on an American Sign Language (ASL) Recognition dataset, the authors implemented the proposed model. The authors achieved 94.8% recognition rate and compared the model with other CNN and traditional algorithms on the same dataset.

show abstract

“…Tao et al (2018) [22] proposed a Convolutional Neural Network (CNN) along with inference fusion and multiview augmentation. This method uses depth images captured by Microsoft Kinect.…”

mentioning

confidence: 99%

Gesture Recognition of RGB and RGB-D Static Images Using Convolutional Neural Networks

Khari¹,

Garg²,

Crespo³

et al. 2019

IJIMAI

View full text Add to dashboard Cite

show abstract

“…In [9], the author presented an ASL alphabet recognition system based on multi-view augmentation and inference fusion using CNN. This method retrieved 3D data from a depth image and created more perspective views for effective training and reduced overfitting.…”

Section: Related Workmentioning

confidence: 99%

Non-Touch Sign Word Recognition Based on Dynamic Hand Gesture Using Hybrid Segmentation and CNN Feature Fusion

2019

View full text Add to dashboard Cite

Hand gesture-based sign language recognition is a prosperous application of human– computer interaction (HCI), where the deaf community, hard of hearing, and deaf family members communicate with the help of a computer device. To help the deaf community, this paper presents a non-touch sign word recognition system that translates the gesture of a sign word into text. However, the uncontrolled environment, perspective light diversity, and partial occlusion may greatly affect the reliability of hand gesture recognition. From this point of view, a hybrid segmentation technique including YCbCr and SkinMask segmentation is developed to identify the hand and extract the feature using the feature fusion of the convolutional neural network (CNN). YCbCr performs image conversion, binarization, erosion, and eventually filling the hole to obtain the segmented images. SkinMask images are obtained by matching the color of the hand. Finally, a multiclass SVM classifier is used to classify the hand gestures of a sign word. As a result, the sign of twenty common words is evaluated in real time, and the test results confirm that this system can not only obtain better-segmented images but also has a higher recognition rate than the conventional ones.

show abstract

“…This task presents a challenging problem that has not yet been solved in computer vision and machine learning. Unlike many previous studies [1,2,3,4,5,6,7], which separately have tried to address hand detection or hand gesture recognition, our approach attempts to jointly solve the problem of hand localization and gesture recognition. This task, however, is very challenging, due to the significant variations of hand images in realistic scenarios.…”

Section: Introductionmentioning

confidence: 99%

A Deep Learning-Based End-to-End Composite System for Hand Detection and Gesture Recognition

Mohammed

Islam

2019

Sensors

View full text Add to dashboard Cite

Recent research on hand detection and gesture recognition has attracted increasing interest due to its broad range of potential applications, such as human-computer interaction, sign language recognition, hand action analysis, driver hand behavior monitoring, and virtual reality. In recent years, several approaches have been proposed with the aim of developing a robust algorithm which functions in complex and cluttered environments. Although several researchers have addressed this challenging problem, a robust system is still elusive. Therefore, we propose a deep learning-based architecture to jointly detect and classify hand gestures. In the proposed architecture, the whole image is passed through a one-stage dense object detector to extract hand regions, which, in turn, pass through a lightweight convolutional neural network (CNN) for hand gesture recognition. To evaluate our approach, we conducted extensive experiments on four publicly available datasets for hand detection, including the Oxford, 5-signers, EgoHands, and Indian classical dance (ICD) datasets, along with two hand gesture datasets with different gesture vocabularies for hand gesture recognition, namely, the LaRED and TinyHands datasets. Here, experimental results demonstrate that the proposed architecture is efficient and robust. In addition, it outperforms other approaches in both the hand detection and gesture classification tasks.

show abstract

American Sign Language alphabet recognition using Convolutional Neural Networks with multiview augmentation and inference fusion

Cited by 130 publications

References 8 publications

Gesture Recognition of RGB and RGB-D Static Images Using Convolutional Neural Networks

Gesture Recognition of RGB and RGB-D Static Images Using Convolutional Neural Networks

Non-Touch Sign Word Recognition Based on Dynamic Hand Gesture Using Hybrid Segmentation and CNN Feature Fusion

A Deep Learning-Based End-to-End Composite System for Hand Detection and Gesture Recognition

Contact Info

Product

Resources

About