Weakly Supervised Training of a Sign Language Recognition System Using Multiple Instance Learning Density Matrices

Kelly, Damien P.; McDonald, John; Markham, Charles H.

doi:10.1109/tsmcb.2010.2065802

Cited by 45 publications

(15 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…MIL iteratively estimates the instance labels measuring a predefined loss. Buehler et al [4] and similarly Kelly et al [17] apply MIL to learning sign categories from TV subtitles, circumventing the translation problem by performing sign spotting. However, Farhadi and Forsyth [9] were the first to approach the subtitle-signalignment problem.…”

Section: State-of-the-artmentioning

confidence: 99%

Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled

Koller

Ney

Bowden

2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

215

150

View full text Add to dashboard Cite

This work presents a new approach to learning a framebased classifier on weakly labelled sequence data by embedding a CNN within an iterative EM algorithm. This allows the CNN to be trained on a vast number of example images when only loose sequence level information is available for the source videos. Although we demonstrate this in the context of hand shape recognition, the approach has wider application to any video recognition task where frame level labelling is not available. The iterative EM algorithm leverages the discriminative ability of the CNN to iteratively refine the frame level annotation and subsequent training of the CNN. By embedding the classifier within an EM framework the CNN can easily be trained on 1 million hand images. We demonstrate that the final classifier generalises over both individuals and data sets. The algorithm is evaluated on over 3000 manually labelled hand shape images of 60 different classes which will be released to the community. Furthermore, we demonstrate its use in continuous sign language recognition on two publicly available large sign language data sets, where it outperforms the current state-of-the-art by a large margin. To our knowledge no previous work has explored expectation maximization without Gaussian mixture models to exploit weak sequence labels for sign language recognition.

show abstract

Section: State-of-the-artmentioning

confidence: 99%

Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled

Koller

Ney

Bowden

2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

215

150

View full text Add to dashboard Cite

show abstract

“…With respect to sign language, several works exist that exploit weak supervision to learn hand-based sign models [20][21][22][23][24]. Facial features have also been used before.…”

Section: State Of the Artmentioning

confidence: 99%

Read My Lips: Continuous Signer Independent Weakly Supervised Viseme Recognition

Koller

Ney

Bowden

2014

Computer Vision – ECCV 2014

View full text Add to dashboard Cite

Abstract. This work presents a framework to recognise signer independent mouthings in continuous sign language, with no manual annotations needed. Mouthings represent lip-movements that correspond to pronunciations of words or parts of them during signing. Research on sign language recognition has focused extensively on the hands as features. But sign language is multi-modal and a full understanding particularly with respect to its lexical variety, language idioms and grammatical structures is not possible without further exploring the remaining information channels. To our knowledge no previous work has explored dedicated viseme recognition in the context of sign language recognition. The approach is trained on over 180.000 unlabelled frames and reaches 47.1% precision on the frame level. Generalisation across individuals and the influence of context-dependent visemes are analysed.

show abstract

“…The traditional SVM model cannot be used to model the temporal changes of the signals, therefore an additional model is required. Over the years, Hidden markov models have been successfully applied in temporal sequence classification applications including speech and gesture recognition [22,23]. Recently, hidden conditional random fields have been proposed as an alternative model for sequence classification and Wang et al [21] have shown that the HCRF model outperforms HMMs.…”

Section: Model Implementationmentioning

confidence: 99%

Automatic detection of collisions in elite level rugby union using a wearable sensing device

et al. 2012

Self Cite

View full text Add to dashboard Cite

Elite rugby union teams currently employ the latest technology to monitor and evaluate the physical demands of training and games on their players. Tackling has been shown to be the most common cause of injury in rugby union, yet current player monitoring technology does not effectively evaluate player tackling measurements. Currently, to evaluate measurements specific to player tackles, a time-consuming manual analysis of player sensor data and video footage is required. The purpose of this work is to investigate tackle modeling techniques which can be utilised to automatically detect player tackles and collisions using sensing technology already being used by elite international and club level rugby union teams. This paper discusses issues relevant to automatic tackle analysis, describes a technique to detect tackles using sensing data and validates the technique by comparing automatically detected collisions to manually labeled collisions using data from elite club and international level players. The results of the validation show that the system is able to consistently identify collisions with very few false positives and false negatives, achieving a recall and precision rating of 0.933 and 0.958, respectively. The aim is that the automatically detected tackles can provide coaching, medical and strength and conditioning staff with objective tackle-specific measurements, in real time, which can be used in injury prevention and rehabilitation strategies.

show abstract

Weakly Supervised Training of a Sign Language Recognition System Using Multiple Instance Learning Density Matrices

Cited by 45 publications

References 27 publications

Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled

Deep Hand: How to Train a CNN on 1 Million Hand Images When Your Data is Continuous and Weakly Labelled

Read My Lips: Continuous Signer Independent Weakly Supervised Viseme Recognition

Automatic detection of collisions in elite level rugby union using a wearable sensing device

Contact Info

Product

Resources

About