Handling Movement Epenthesis and Hand Segmentation Ambiguities in Continuous Sign Language Recognition Using Nested Dynamic Programming

Abstract

Abstract-We consider two crucial problems in continuous sign language recognition from unaided video sequences. At the sentence level, we consider the movement epenthesis (me) problem and at the feature level, we consider the problem of hand segmentation and grouping. We construct a framework that can handle both of these problems based on an enhanced, nested version of the dynamic programming approach. To address movement epenthesis, a dynamic programming (DP) process employs a virtual me option that does not need explicit models. We call this the enhanced level building (eLB) algorithm. This formulation also allows the incorporation of grammar models. Nested within this eLB is another DP that handles the problem of selecting among multiple hand candidates. We demonstrate our ideas on four American Sign Language data sets with simple background, with the signer wearing short sleeves, with complex background, and across signers. We compared the performance with Conditional Random Fields (CRF) and Latent Dynamic-CRF-based approaches. The experiments show more than 40 percent improvement over CRF or LDCRF approaches in terms of the frame labeling rate. We show the flexibility of our approach when handling a changing context. We also find a 70 percent improvement in sign recognition rate over the unenhanced DP matching algorithm that does not accommodate the me effect.

Implementation of Hand Sign Recognition for Non-Linear Dimensionality Reduction based on PCA

“…Similarly count two ‗means reverse', count three means ‗turn right', count four means ‗turn left'. These are the terms used in robot control with hand gestures [13].…”

“…Process which separates object and background of image is called segmentation. In this above paper segmentation errors are removed by dynamic programming [13].…”

An Enhanced Framework for Sign Gesture Recognition using Hidden Markov Model and Adaptive Histogram Technique

“…Signal investigation is critical for various application fields, since it is worried with the communications amongst people, and amongst people and nature or machines [9]. These applications for the most part require the improvement of strategies for distinguishing developments made by clients, recognizing the fragments of the development that worry the application and perceiving the development inside a scope of expected developments [10]. With a specific end goal to fabricate these applications, distinctive systems can be utilized for tackling each previously mentioned assignment [11,12].…”

Recognitionwith raw canonical phonetic movement and handshape subunits on videos of continuous Sign Language

“…Multiple cues are combined in [10] for isolated sign recognition. Another aspect concerns continuous SLR [16,18,19,20] and issues such as coarticulation and movement epenthesis. Nested dynamic programming is employed in [18] to handle movement epenthesis.…”

“…Another aspect concerns continuous SLR [16,18,19,20] and issues such as coarticulation and movement epenthesis. Nested dynamic programming is employed in [18] to handle movement epenthesis. Transition-movement models are employed for large-vocabulary continuous SLR [19]; [20] presents a threshold model based on conditional random fields.…”