Unsupervised Modeling of Signs Embedded in Continuous Sentences

Chen

2006 IEEE International Conference on Multimedia and Expo

et al. 2006

It is difficult to recognize sign language in different viewpoint. The HMM method is hindered by the difficulty of extracting view invariant features. The general template matching methods have a strong constraint such as accurate alignment between the template sign and the test sign. In the paper, we introduce a novel approach for viewpoint invariant sign language recognition. The proposed approach requires no view invariant features, low training and no alignment. Its basic idea is to consider a sign as a series of tiny hand motions and utilize the HOMOGRAPHY of tiny hand motions. Using the word of "homography", we mean that there are the same tiny hand motions as well as their appearance order in different performances of the same sign. The experimental results demonstrate the efficiency of the proposed method.

Section: Introductionmentioning

confidence: 99%

Sign Language Recognition from Homography

Chen

2006 IEEE International Conference on Multimedia and Expo

et al. 2006

2009 IEEE Conference on Computer Vision and Pattern Recognition

“…Signeme represents the portion of the sign that is most similar across the sentences [17]. We formulate the signeme extraction problem as finding the most recurring pattern among a set of n sentences { S 1 , · · · , S n }, that have one common sign present in all the sentences.…”

Section: Problem Formulationmentioning

confidence: 99%

“…Some of the motif discovery works illustrating results on human movements include [10,16,1]. Nayak et al [17] find recurrent patterns from sign language sentences. Yang et al [24] perform sign spotting in continuous sign language sentences using Conditional Random Fields.…”

Section: Introductionmentioning

confidence: 99%

“…Our current work can be used to automatically generate such training data, cutting down on the tedious nature of the process. Nayak et al [17] proposed a sequential method for extracting signeme models from continuous sign language sentences. But the signemes extracted were heavily biased by the first two sentences used to start the search.…”

Section: Introductionmentioning

confidence: 99%

“…We assume that the sign to be extracted is the only sign that is common to each of the n given sentences. Skin color blobs are extracted from frames of color video, and a relational distribution [17] is formed for each frame using the edge pixels in the skin blobs. Each sentence is then represented as a trajectory in a low dimensional space called Space of Relational Distributions, which is arrived at by performing Principal Component Analysis (PCA) on the relational distributions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automated extraction of signs from continuous sign language sentences using Iterated Conditional Modes

Nayak

Sarkar

Loeding

2009

The Springer Series on Challenges in Machine Learning

Challenges in Multi-modal Gesture Recognition

Escalera

Athitsos

Guyon

2017

This paper surveys the state of the art on multimodal gesture recognition and introduces the JMLR special topic on gesture recognition 2011-2015. We began right at the start of the Kinect T M revolution when inexpensive infrared cameras providing image depth recordings became available. We published papers using this technology and other more conventional methods, including regular video cameras, to record data, thus providing a good overview of uses of machine learning and computer vision using multimodal data in this area of application. Notably, we organized a series of challenges and made available several datasets we recorded for that purpose, including tens of thousands of videos, which are available to conduct further research. We also overview recent state of the art works on gesture recognition based on a proposed taxonomy for gesture recognition, discussing challenges and future lines of research.