American Sign Language Posture Understanding with Deep Neural Networks

Jalal, Asif; Chen, Ruilong; Moore, Roger K.; Mihaylova, Lyudmila

doi:10.23919/icif.2018.8455725

Cited by 30 publications

(15 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, the network trainable parameters are reduced with respect to other deep learning architectures, confirming the architectural advantage given by the introduced features also in the task of polyphonic SED. The results we observed in this work are consistent with many other classification tasks in various domains [44]- [46] and they prove that the CapsNet is an effective approach which enhances the well-established representation capabilities of the CNNs also in the audio field. However, several aspects still remain unexplored and require further studies: the robustness of CapsNets to overlapping signals (i.e., images or sounds) has been demonstrated in this work as well as in [23].…”

Section: Discussionsupporting

confidence: 88%

Polyphonic Sound Event Detection by Using Capsule Neural Networks

Vesperini

Gabrielli

Principi

et al. 2019

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Artificial sound event detection (SED) has the aim to mimic the human ability to perceive and understand what is happening in the surroundings. Nowadays, learning offers valuable techniques for this goal such as convolutional neural networks (CNNs). The capsule neural network (CapsNet) architecture has been recently introduced in the image processing field with the intent to overcome some of the known limitations of CNNs, specifically regarding the scarce robustness to affine transformations (i.e., perspective, size, orientation) and the detection of overlapped images. This motivated the authors to employ CapsNets to deal with the polyphonic SED task, in which multiple sound events occur simultaneously. Specifically, we propose to exploit the capsule units to represent a set of distinctive properties for each individual sound event. Capsule units are connected through a so-called dynamic routing that encourages learning part-whole relationships and improves the detection performance in a polyphonic context. This paper reports extensive evaluations carried out on three publicly available datasets, showing how the CapsNet-based algorithm not only outperforms standard CNNs but also allows to achieve the best results with respect to the state-of-the-art algorithms.

show abstract

Section: Discussionsupporting

confidence: 88%

Polyphonic Sound Event Detection by Using Capsule Neural Networks

Vesperini

Gabrielli

Principi

et al. 2019

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Sign language is utilized as the first language by millions of deaf people (hearing impaired people), in addition, hard-of-hearing people, and people who have various speaking difficulties. In accordance with the investigation conducted by the British Deaf Association, it is recorded that about 151,000 people use sign language as a means of communication [1]. There is no universal sign language and almost every country has its own national sign language and fingerspelling alphabet.…”

Section: Introductionmentioning

confidence: 75%

Sign Language Translation Using Deep Convolutional Neural Networks

2020

KSII TIIS

View full text Add to dashboard Cite

Sign language is a natural, visually oriented and non-verbal communication channel between people that facilitates communication through facial/bodily expressions, postures and a set of gestures. It is basically used for communication with people who are deaf or hard of hearing. In order to understand such communication quickly and accurately, the design of a successful sign language translation system is considered in this paper. The proposed system includes object detection and classification stages. Firstly, Single Shot Multi Box Detection (SSD) architecture is utilized for hand detection, then a deep learning structure based on the Inception v3 plus Support Vector Machine (SVM) that combines feature extraction and classification stages is proposed to constructively translate the detected hand gestures. A sign language fingerspelling dataset is used for the design of the proposed model. The obtained results and comparative analysis demonstrate the efficiency of using the proposed hybrid structure in sign language translation.

show abstract

“…This measure considers false positives and false negatives both. Classification's overall accuracy is measured using following equation: Overall Accuracy = summation of all diagonal elements / Total no of elements (1) Precision based on class (1-10) is measured using following equation: Precision = Diagonal elements of row /other elements in row =TP/TP+FP (2) Recall is measured using following equation: Recall = TP/TP+FN…”

Section: Experiments Results Analysismentioning

confidence: 99%

“…Sign Language Recognition is a multifaceted task that makes use of body expression detection such as hand and facial movement detection. This detection mechanism is based on various features such as shape and motion features which forms the essential structural blocks of the recognition system [1] [2]. Among the various body movements and gestures, the hand gestures are most widely used as a medium of sign language based communication framework.…”

Section: Introductionmentioning

confidence: 99%

“…Bag of feature extraction method improve the efficiency compared to HOG method. Use of Deep learning [1][2][9] [10] has tremendously increased in the area of image classification, natural language processing ,image recognition, speech recognition and many more. Several methodologies of image processing have been applied for classification of Indian sign language.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Feature Extraction using Pre-Trained Alex Net Model for Indian Sign Language Recognition

Dangarwala¹,

Hiran²

2019

IJRTE

View full text Add to dashboard Cite

Indian sign language is communicating language among deaf and dumb people of India. Hand gestures are broadly used as communication gestures among various forms of gesture. The real time classification of different signs is a challenging task due to the variation in shape and position of hands as well as due to the variation in the background which varies from person to person. There seems to be no availability of datasets resembling to Indian signs which poses a problem to the researcher. To address this problem, we design our own dataset which is formed by incorporating 1000 signs for the sign digits from 1 to 10 from 100 different people with varying backgrounds conditions by changing colour, and light illumination situations. The dataset comprises of the signs from left handed as well as right handed people. Feature extraction methodologies are studied and applied to recognition of Sign language. This paper focuses on deep learning CNN (convolution neural network) approach with pretrained model Alexnet for calculation of feature vector. Multiple SVM (Support Vector Machine) is applied to classify Indian sign language in real time surroundings. This paper also shows the comparative analysis between Deep learning feature extraction method with histogram of gradient, bag of feature and Speed up robust feature extraction method. The experimental results shown that Deep learning feature extraction using pretrained Alexnet model give accuracy of around 85% and above for the recognition of signed digit with the use of 60% training set and 40% testing set.

show abstract

American Sign Language Posture Understanding with Deep Neural Networks

Abstract: This is a repository copy of American sign language posture understanding with deep neural networks.

Cited by 30 publications

References 21 publications

Polyphonic Sound Event Detection by Using Capsule Neural Networks

Polyphonic Sound Event Detection by Using Capsule Neural Networks

Sign Language Translation Using Deep Convolutional Neural Networks

Deep Learning Feature Extraction using Pre-Trained Alex Net Model for Indian Sign Language Recognition

Contact Info

Product

Resources

About