Motionlets Matching With Adaptive Kernels for 3-D Indian Sign Language Recognition

Kishore, P. V. V.; Kumar, D. Anil; Sastry, Ananth; Kumar, E. Kiran

doi:10.1109/jsen.2018.2810449

Cited by 74 publications

(31 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this is first time a 2D skeletal sign language video data is being used for multi view recognition. The 2D skeletal videos are generated for 3D sign language data captured with 3D motion capture technology [6,7]…”

Section: Problem Statementmentioning

confidence: 99%

See 1 more Smart Citation

Learning multiview deep features from skeletal sign language videos for recognition

Shaik¹,

Mareedu²,

Kishore³

2021

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

The most challenging objective in machine translation of sign language has been the machine's inability to learn inter occluding finger movements during an action process. This work addresses the problem of teaching a deep learning model to recognize differently oriented skeletal data. The multi view 2D skeletal sign language video data is obtained using 3D motion captured system. Total 9 views of the signer were used for training the proposed network and the remaining 6 for testing and validation. In order to obtain multi view deep features for recognition, we proposed an end-to-end trainable multi stream convolutional neural network (CNN) with late feature fusion. The fused multi view features are then inputted to a two-layer dense and a decision making SoftMax. The proposed CNN employs numerous layers to characterize view correspondence to generate maximally discriminative features. This study is important to understand the effects of multi view data processing by CNNs for sign language recognition in decoding joint spatial information. Further, deeper perspectives were developed into multi view processing of CNNs by applying skeletal action data.

show abstract

Section: Problem Statementmentioning

confidence: 99%

“…Consequently, the 3D sensors such as Kinect [4], leap motion [5] and motion capture [6] has bettered the SLR results in the past decade. However, only motion capture had the capability to model near perfect biomechanics of human body to recreate sign language 3D data [7].…”

Section: Introductionmentioning

confidence: 99%

Learning multiview deep features from skeletal sign language videos for recognition

Shaik¹,

Mareedu²,

Kishore³

2021

Turk J Elec Eng & Comp Sci

View full text Add to dashboard Cite

show abstract

“…The characterization to articulate sign language gestures in different parts of the body as three-dimensional movements has been presented [12].In [13] Rajam et al proposed recognizition of South Indian Sign Language gestures by considering 32 number of gestures with an obtained accuracy of 98.12%. In [14], the work has been performed by Deora et al to recognize ISL gesture by considering both hand and overlapping hand.…”

Section: Related Workmentioning

confidence: 99%

An Efficient Indian Sign Language Recognition System using Sift Descriptor

Kaur¹,

Krishna²

2019

IJEAT

View full text Add to dashboard Cite

Communication is one of the basic requirements for living in the world. Deaf and dumb people convey through Sign Language but normal people have difficulty to understand their language. In order to provide a source of medium between normal and differently abled people, Sign Language Recognition System (SLR) is a solution . American Sign Language (ASL) has attracted many researchers’ attention but Indian Sign Language Recognition (ISLR) is significantly different from ASL due to different phonetic, grammar and hand movement. Designing a system for Indian Sign Language Recognition becomes a difficult task. ISLR system uses Indian Sign Language (ISL) dataset for recognition but suffers from problem of scaling, object orientation and lack of optimal feature set. In this paper to address these issues, Scale-Invariant Feature Transform (SIFT) as a descriptor is used. It extracts the features that train the Feed Forward Back Propagation Neural Network (FFBPNN) and optimize it using Artificial Bee Colony (ABC) according to the fitness function. The dataset has been collected for alphabet from the video by extracting frames and for numbers it has been created manually from deaf and dumb students of NGO “Sarthak”. It has been shown through simulation results that there has been significant improvement in accurately identifying alphabets and numbers with an average accuracy of 99.43%

show abstract

“…Features such as 3D graph joint trajectory locations [12] and joint relative distances [13] are used for human motion analysis. The features form human actions are classified using support vector machine [14], convolutional Neural Networks [15], Dynamic Time Warping [16], weighted graph matching [17] and Histogram [18]. However, JRD's and RRJRD's based descriptors for human action recognition were successfully used with graph kernel matching in [13] [14].…”

Section: Literature Reviewmentioning

confidence: 99%

Spatial Joint features for 3D human skeletal action recognition system using spatial graph kernels

Kishore¹,

Kameswari²,

Niharika³

et al. 2017

IJET

View full text Add to dashboard Cite

Human action recognition is a vibrant area of research with multiple application areas in human machine interface. In this work, we propose a human action recognition based on spatial graph kernels on 3D skeletal data. Spatial joint features are extracted using joint distances between human joint distributions in 3D space. A spatial graph is constructed using 3D points as vertices and the computed joint distances as edges for each action frame in the video sequence. Spatial graph kernels between the training set and testing set are constructed to extract similarity between the two action sets. Two spatial graph kernels are constructed with vertex and edge data represented by joint positions and joint distances. To test the proposed method, we use 4 publicly available 3D skeletal datasets from G3D, MSR Action 3D, UT Kinect and NTU RGB+D. The proposed spatial graph kernels result in better classification accuracies compared to the state of the art models.

show abstract

Motionlets Matching With Adaptive Kernels for 3-D Indian Sign Language Recognition

Cited by 74 publications

References 39 publications

Learning multiview deep features from skeletal sign language videos for recognition

Learning multiview deep features from skeletal sign language videos for recognition

An Efficient Indian Sign Language Recognition System using Sift Descriptor

Spatial Joint features for 3D human skeletal action recognition system using spatial graph kernels

Contact Info

Product

Resources

About