Nonlinear manifold learning for visual speech recognition

Bregler, Christoph; Omohundro, Stephen M.

doi:10.1109/iccv.1995.466899

Cited by 111 publications

(88 citation statements)

References 14 publications

(9 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Automatic dimension detection is a major challenge in the fields of learning theory, pattern recognition [5] and artificial intelligence in general [20,23]. In these applications samples can be generated from an otherwise unknown manifold.…”

Section: Introductionmentioning

confidence: 99%

Shape Dimension and Approximation from Samples

Dey¹,

Giesen²,

Goswami³

et al. 2003

Discrete and Computational Geometry

View full text Add to dashboard Cite

There are many scientific and engineering applications where an automatic detection of shape dimension from sample data is necessary. Topological dimensions of shapes constitute an important global feature of them. We present a Voronoi based dimension detection algorithm that assigns a dimension to a sample point which is the topological dimension of the manifold it belongs to, Based on this dimension detection, we also present an algorithm to approximate shapes of arbitrary dimension from their samples. Our empirical results with data sets in three dimensions support our theory.

show abstract

Section: Introductionmentioning

confidence: 99%

Shape Dimension and Approximation from Samples

Dey¹,

Giesen²,

Goswami³

et al. 2003

Discrete and Computational Geometry

View full text Add to dashboard Cite

show abstract

“…In this sense, the hidden states can follow the manifold and, therefore, HMMs model the observation manifolds in implicit ways, e.g. as in [30] and in [31].…”

Section: Manifold-based Models Of Human Motionmentioning

confidence: 99%

Homeomorphic Manifold Analysis (HMA): Generalized separation of style and content on manifolds

Elgammal

Lee

2013

Image and Vision Computing

View full text Add to dashboard Cite

The problem of separation of style and content is an essential element of visual perception, and is a fundamental mystery of perception. This problem appears extensively in different computer vision applications. The problem we address in this paper is the separation of style and content when the content lies on a low dimensional nonlinear manifold representing a dynamic object. We show that such a setting appears in many human motion analysis problems. We introduce a framework for learning parameterization of style and content in such settings. Given a set of topologically equivalent manifolds, the Homeomorphic Manifold Analysis (HMA) framework models the variation in their geometries in the space of functions that maps between a topologically-equivalent common representation and each of them. The framework is based on decomposing the style parameters in the space of nonlinear functions that map between a unified embedded representation of the content manifold and style-dependent visual observations. We show the application of the framework in synthesis, recognition, and tracking of certain human motions that follow this setting, such as gait and facial expressions.

show abstract

“…Furthermore, temporal information between the channels is lost in this approach. AVSR systems based on EI models have for example been described in 6,32] and systems based on LI models in 27,30]. Although it is still not well known how h umans integrate di erent modalities, it is generally agreed that integration occurs before speech is categorised phonetically 5, 3 1 ].…”

Section: Audio-visual Sensor Integrationmentioning

confidence: 99%

Continuous audio-visual speech recognition

Luettin

Dupont

1998

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We address the problem of robust lip tracking, visual speech feature extraction, and sensor integration for audio-visual speech recognition applications. An appearance based model of the articulators, which represents linguistically important features, is learned from example images and is used to locate, track, and recover visual speech information. We t a c kle the problem of joint temporal modelling of the acoustic and visual speech signals by applying Multi-Stream hidden Markov models. This approach allows the use of di erent temporal topologies and levels of stream integration and hence enables to model temporal dependencies more accurately. T h e system has been evaluated for a continuously spoken digit recognition task of 37 subjects. IDIAP{RR 98-02

show abstract

Nonlinear manifold learning for visual speech recognition

Cited by 111 publications

References 14 publications

Shape Dimension and Approximation from Samples

Shape Dimension and Approximation from Samples

Homeomorphic Manifold Analysis (HMA): Generalized separation of style and content on manifolds

Continuous audio-visual speech recognition

Contact Info

Product

Resources

About