Modelling and segmenting subunits for sign language recognition based on hand motion analysis

Han, Junwei; Awad, George; Sutherland, Alistair

doi:10.1016/j.patrec.2008.12.010

Cited by 79 publications

(53 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We can see that the best result for subjects 21-25 were 76% with 0.70 SIFT threshold, 80% with 0.70 SIFT threshold, 66% with 0.70 SIFT threshold, 68% with 0.65 and 0.70 SIFT threshold, and 62% with 0.75 SIFT threshold for the five subjects, respectively. Since the signers of this test set (subjects [21][22][23][24][25] were different signers from the training data set and the signature library, the results of this experiment provided low classification. Furthermore, when we used SIFT with the unconstrained system and complex natural backgrounds, the matched keypoints might be incorrectly matched, as shown Figure 3g.…”

Section: Resultsmentioning

confidence: 99%

“…This allows the system to be able to recognise hand sign words that have similar gestures. However, when we tested our algorithm with the test subject without any constraint on five signers (subjects [21][22][23][24][25], who were asked to stand in front of various complex backgrounds and could wear any shirt, the best correct classification rate in this case was around 70-80% on average.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

2017

View full text Add to dashboard Cite

Abstract:Sign language is a basic method for solving communication problems between deaf and hearing people. In order to communicate, deaf and hearing people normally use hand gestures, which include a combination of hand positioning, hand shapes, and hand movements. Thai Sign Language is the communication method for Thai hearing-impaired people. Our objective is to improve the dynamic Thai Sign Language translation method with a video captioning technique that does not require prior hand region detection and segmentation through using the Scale Invariant Feature Transform (SIFT) method and the String Grammar Unsupervised Possibilistic C-Medians (sgUPCMed) algorithm. This work is the first to propose the sgUPCMed algorithm to cope with the unsupervised generation of multiple prototypes in the possibilistic sense for string data. In our experiments, the Thai Sign Language data set (10 isolated sign language words) was collected from 25 subjects. The best average result within the constrained environment of the blind test data sets of signer-dependent cases was 89-91%, and the successful rate of signer semi-independent cases was 81-85%, on average. For the blind test data sets of signer-independent cases, the best average classification rate was 77-80%. The average result of the system without a constrained environment was around 62-80% for the signer-independent experiments. To show that the proposed algorithm can be implemented in other sign languages, the American sign language (RWTH-BOSTON-50) data set, which consists of 31 isolated American Sign Language words, is also used in the experiment. The system provides 88.56% and 91.35% results on the validation set alone, and for both the training and validation sets, respectively.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

2017

View full text Add to dashboard Cite

show abstract

“…Han et al [8] perform a segmentation of data based on linguistic rules, such as change of hand motion and discontinuities surrounding the subunit boundary.…”

Section: Related Workmentioning

confidence: 99%

May the force be with you: Force-aligned signwriting for automatic subunit annotation of corpora

Koller

Ney

Bowden

2013

2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)

View full text Add to dashboard Cite

Abstract-We propose a method to generate linguistically meaningful subunits in a fully automated fashion for sign language corpora. The ability to automate the process of subunit annotation has profound effects on the data available for training sign language recognition systems. The approach is based on the idea that subunits are shared among different signs. With sufficient data and knowledge of possible signing variants, accurate automatic subunit sequences are produced, matching the specific characteristics of given sign language data.Specifically we demonstrate how an iterative forced alignment algorithm can be used to transfer the knowledge of a useredited open sign language dictionary to the task of annotating a challenging, large vocabulary, multi-signer corpus recorded from public TV. Existing approaches focus on labour intensive manual subunit annotations or on data-driven approaches. Our method yields an average precision and recall of 15% under the maximum achievable accuracy with little user intervention beyond providing a simple word gloss.

show abstract

“…Yin et al [110] used an accelerometer glove to gather information about a sign, before applying discriminative feature extraction and similar state tying algorithms, to decide sub-unit level segmentation of the data. Kong et al [58] and Han et al [41] have looked at automatic segmentation of the motions of sign into sub-units, using discontinuities in the trajectory and acceleration, to indicate where segments begin and end, these are then clustered into a code book of possible exemplar trajectories using either DTW distance measures, in the case of Han et al or PCA features by Kong et al…”

Section: Phoneme Level Representationsmentioning

confidence: 99%

Sign Language Recognition

Cooper

Holt

Bowden

2011

Visual Analysis of Humans

176

View full text Add to dashboard Cite

This chapter covers the key aspects of Sign Language Recognition (SLR), starting with a brief introduction to the motivations and requirements, followed by a précis of sign linguistics and their impact on the field. The types of data available and the relative merits are explored allowing examination of the features which can be extracted. Classifying the manual aspects of sign (similar to gestures) is then discussed from a tracking and non-tracking viewpoint before summarising some of the approaches to the non-manual aspects of sign languages. Methods for combining the sign classification results into full SLR are given showing the progression towards speech recognition techniques and the further adaptations required for the sign specific case. Finally the current frontiers are discussed and the recent research presented. This covers the task of continuous sign recognition, the work towards true signer independence, how to effectively combine the different modalities of sign, making use of the current linguistic research and adapting to larger more noisy data sets.

show abstract

Modelling and segmenting subunits for sign language recognition based on hand motion analysis

Cited by 79 publications

References 9 publications

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

May the force be with you: Force-aligned signwriting for automatic subunit annotation of corpora

Sign Language Recognition

Contact Info

Product

Resources

About