Sign Language Recognition Using Sub-units

Cooper, Helen; Ong, Eng-Jon; Pugeault, Nicolas; Bowden, Richard

doi:10.1007/978-3-319-57021-1_3

Cited by 84 publications

(84 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Another branch of sign language recognition research focuses instead on classification of individually segmented signs. One popular intuitive method is to segment a sign into motion or other types of sub-units and then use an HMM to model the temporal changes in sub-units throughout each sign [28], [29]. Dynamic Time Warping has also been used for action and gesture recognition [7], [8], [30], [31] and it has shown its superiority over LSTM and HMM models [31].…”

Section: Related Workmentioning

confidence: 99%

Local frame match distance: A novel approach for exemplar gesture recognition

Ionescu

Popescu

Conly

et al. 2017

2017 25th European Signal Processing Conference (EUSIPCO)

View full text Add to dashboard Cite

Abstract-Gesture recognition using a training set of limited size for a large vocabulary of gestures is a challenging problem in computer vision. With few examples per gesture class, researchers often employ state-of-the-art exemplar-based methods such as Dynamic Time Warping (DTW). This paper makes two contributions in the area of exemplar-based gesture recognition. As an alternative to DTW, we first introduce the Local Frame Match Distance (LFMD), a novel approach for matching gestures inspired by a distance measure for strings, namely Local Rank Distance (LRD). While LRD efficiently approximates the non-alignment of character n-grams between two strings, we employ LFMD to efficiently measure the nonalignment of hand locations between two video sequences. Second of all, we transform LFMD into a kernel and use it in combination with Kernel Discriminant Analysis for sign language recognition with exemplars. The empirical results indicate that our method can generally yield better performance than a state-of-the-art DTW approach on the challenging task of American Sign Language recognition, while reducing the computational time by 30%.

show abstract

Section: Related Workmentioning

confidence: 99%

Local frame match distance: A novel approach for exemplar gesture recognition

Ionescu

Popescu

Conly

et al. 2017

2017 25th European Signal Processing Conference (EUSIPCO)

View full text Add to dashboard Cite

show abstract

“…Zafrulla et al [5] used the PrimeSense tracker to perform sentence recognition on a dataset of ASL sentences from an educational game called CopyCat [26]. Copper et al [27] used the PrimeSense tracker to perform linguistic sub-unit-based sign language recognition on signs from German Sign Language.…”

Section: Related Workmentioning

confidence: 99%

Hand detection in American Sign Language depth data using domain-driven random forest regression

Zafrulla

Sahni

Bedri

et al. 2015

2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)

View full text Add to dashboard Cite

In Automatic Sign Language Recognition (ASLR), robust hand tracking and detection is key to good recognition accuracy. We introduce a new dataset of depth data from continuously signed American Sign Language (ASL) sentences. We present analysis showing numerous errors of the Microsoft Kinect Skeleton Tracker (MKST) in cases where hands are close to the body, close to each other, or when the arms cross. We also propose a method based on domain-driven random forest regression, which predicts real world 3D hand locations using features generated from depth images. We show that our hand detector (DDRFR) has >20% improvement over the MKST within a margin of error of 5 cm from the ground truth.

show abstract

“…In these approaches, spatial features are grouped into the building blocks of gestures, such as states [11] or subunits [12], and changes among these states are modeled using graphical models. Since the pioneering work of Starner and Pentland [3], Hidden Markov Models have often been used for gesture recognition [13], [14], [15].…”

Section: Introductionmentioning

confidence: 99%

Using Convolutional 3D Neural Networks for User-independent continuous gesture recognition

Camgöz

Hadfield

Koller

et al. 2016

2016 23rd International Conference on Pattern Recognition (ICPR)

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, we propose using 3D Convolutional Neural Networks for large scale user-independent continuous gesture recognition. We have trained an end-to-end deep network for continuous gesture recognition (jointly learning both the feature representation and the classifier). The network performs three-dimensional (i.e. space-time) convolutions to extract features related to both the appearance and motion from volumes of color frames. Space-time invariance of the extracted features is encoded via pooling layers. The earlier stages of the network are partially initialized using the work of Tran et al. before being adapted to the task of gesture recognition. An earlier version of the proposed method, which was trained for 11,250 iterations, was submitted to ChaLearn 2016 Continuous Gesture Recognition Challenge and ranked 2nd with the Mean Jaccard Index Score of 0.269235. When the proposed method was further trained for 28,750 iterations, it achieved state-of-the-art performance on the same dataset, yielding a 0.314779 Mean Jaccard Index Score.

show abstract

Sign Language Recognition Using Sub-units

Cited by 84 publications

References 25 publications

Local frame match distance: A novel approach for exemplar gesture recognition

Local frame match distance: A novel approach for exemplar gesture recognition

Hand detection in American Sign Language depth data using domain-driven random forest regression

Using Convolutional 3D Neural Networks for User-independent continuous gesture recognition

Contact Info

Product

Resources

About