Tracking Using Dynamic Programming for Appearance-Based Sign Language Recognition

Dreuw, Philippe; Deselaers, Thomas; Rybach, David; Keysers, Daniel; Ney, Hermann

doi:10.1109/fgr.2006.107

Cited by 38 publications

(30 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Here we extended the tracking framework proposed in [1], by using the obtained depth information not only as features for the models to be trained but also as scoring function q(u t−1 , u t ; x t t−1 ) to determine a likelihood for the tracked hand being still the correct one. In particular, after hands were overlapping, the tracker often confused the hands afterwards (c.f.…”

Section: Extending Hand Tracking With Stereo Featuresmentioning

confidence: 99%

Smoothed Disparity Maps for Continuous American Sign Language Recognition

Dreuw

Steingrube

Deselaers

et al. 2009

Pattern Recognition and Image Analysis

View full text Add to dashboard Cite

Abstract. For the recognition of continuous sign language we analyse whether we can improve the results by explicitly incorporating depth information. Accurate hand tracking for sign language recognition is made difficult by abrupt and fast changes in hand position and configuration, overlapping hands, or a hand signing in front of the face. In our system depth information is extracted using a stereo-vision method that considers the time axis by using pre-and succeeding frames. We demonstrate that depth information helps to disambiguate overlapping hands and thus to improve the tracking of the hands. However, the improved tracking has little influence on the final recognition results.

show abstract

Section: Extending Hand Tracking With Stereo Featuresmentioning

confidence: 99%

Smoothed Disparity Maps for Continuous American Sign Language Recognition

Dreuw

Steingrube

Deselaers

et al. 2009

Pattern Recognition and Image Analysis

View full text Add to dashboard Cite

show abstract

“…Therefore, a robust tracking algorithm for hand tracking is required. Instead of tracking by detection, it is also possible to optimize the tracking decision considering the full sequence using dynamic programming (DP) [3]. This has the advantage to reduce tracking errors and the structure of the algorithm allows for fully integrating this into the recognition process (c.f.…”

Section: Hand Trackingmentioning

confidence: 99%

“…In contrast to these works, here we work on the recognition of continuous sign language. For the recognition of sign language the hand is the part of the image that is moving most [3,18]. Most approaches addressing the recognition of gestures and sign language use a two-step procedure, where in the first step the hand is tracked and in the second step the classification recognition is done [7,11].…”

Section: Introductionmentioning

confidence: 99%

Efficient approximations to model-based joint tracking and recognition of continuous sign language

Dreuw

Förster

Deselaers

et al. 2008

2008 8th IEEE International Conference on Automatic Face &Amp; Gesture Recognition

View full text Add to dashboard Cite

We propose several tracking adaptation approaches to recover from early tracking errors in sign language recognition by optimizing the obtained tracking paths w.r.t. to the hypothesized word sequences of an automatic sign language recognition system. Hand or head tracking is usually only optimized according to a tracking criterion. As a consequence, methods which depend on accurate detection and tracking of body parts lead to recognition errors in gesture and sign language processing. We analyze an integrated tracking and recognition approach addressing these problems and propose approximation approaches over multiple hand hypotheses to ease the time complexity of the integrated approach. Most state-of-the-art systems consider tracking as a preprocessing feature extraction part. Experiments on a publicly available benchmark database show that the proposed methods strongly improve the recognition accuracy of the system.

show abstract

“…They then combined skin segmentation with five types of differencing for each frame in a sequence, all are down sampled to obtain features [113]. Following this, their appearance based features were combined with the tracking work of Dreuw et al [24] and some geometric features in the form of moments. Creating a system which fuses both tracking and non-tracking based approaches [111].…”

Section: Non-tracking Basedmentioning

confidence: 99%

“…While adaptable to real time use, it suffers from the same problems as other colour only based approaches. Dreuw et al used dynamic programming to determine the path of the head and hands along a whole video sequence, avoiding such failures at the local level [24] but negating the possibility of real-time application.…”

Section: Tracking Basedmentioning

confidence: 99%

Sign Language Recognition

Cooper

Holt

Bowden

2011

Visual Analysis of Humans

176

View full text Add to dashboard Cite

This chapter covers the key aspects of Sign Language Recognition (SLR), starting with a brief introduction to the motivations and requirements, followed by a précis of sign linguistics and their impact on the field. The types of data available and the relative merits are explored allowing examination of the features which can be extracted. Classifying the manual aspects of sign (similar to gestures) is then discussed from a tracking and non-tracking viewpoint before summarising some of the approaches to the non-manual aspects of sign languages. Methods for combining the sign classification results into full SLR are given showing the progression towards speech recognition techniques and the further adaptations required for the sign specific case. Finally the current frontiers are discussed and the recent research presented. This covers the task of continuous sign recognition, the work towards true signer independence, how to effectively combine the different modalities of sign, making use of the current linguistic research and adapting to larger more noisy data sets.

show abstract

Tracking Using Dynamic Programming for Appearance-Based Sign Language Recognition

Cited by 38 publications

References 14 publications

Smoothed Disparity Maps for Continuous American Sign Language Recognition

Smoothed Disparity Maps for Continuous American Sign Language Recognition

Efficient approximations to model-based joint tracking and recognition of continuous sign language

Sign Language Recognition

Contact Info

Product

Resources

About