ML estimation of a stochastic linear system with the EM algorithm and its application to speech recognition

Digalakis, Vassilios; Rohlíček, Ján; Ostendorf, Mari

doi:10.1109/89.242489

Cited by 207 publications

(161 citation statements)

References 26 publications

Supporting

Mentioning

161

Contrasting

Order By: Relevance

“…A preliminary version of our work appeared in [8]- [10]. In single modality, audio-only scenarios, modeling audio feature noise has proven fruitful for noise-robust ASR [11]- [14] and also in applications such as speaker verification [15] and multiband ASR [16]; see [17] for further pointers to the related literature. We extend these ideas to the multimodal setting and show in Section II how multi-stream classification rules should be adjusted to compensate for feature measurement uncertainty.…”

mentioning

confidence: 99%

Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition

Papandreou

Katsamanis

Pitsikalis

et al. 2009

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Abstract-While the accuracy of feature measurements heavily depends on changing environmental conditions, studying the consequences of this fact in pattern recognition tasks has received relatively little attention to date. In this paper, we explicitly take feature measurement uncertainty into account and show how multimodal classification and learning rules should be adjusted to compensate for its effects. Our approach is particularly fruitful in multimodal fusion scenarios, such as audiovisual speech recognition, where multiple streams of complementary time-evolving features are integrated. For such applications, provided that the measurement noise uncertainty for each feature stream can be estimated, the proposed framework leads to highly adaptive multimodal fusion rules which are easy and efficient to implement. Our technique is widely applicable and can be transparently integrated with either synchronous or asynchronous multimodal sequence integration architectures. We further show that multimodal fusion methods relying on stream weights can naturally emerge from our scheme under certain assumptions; this connection provides valuable insights into the adaptivity properties of our multimodal uncertainty compensation approach. We show how these ideas can be practically applied for audiovisual speech recognition. In this context, we propose improved techniques for person-independent visual feature extraction and uncertainty estimation with active appearance models, and also discuss how enhanced audio features along with their uncertainty estimates can be effectively computed. We demonstrate the efficacy of our approach in audiovisual speech recognition experiments on the CUAVE database using either synchronous or asynchronous multimodal integration models.Index Terms-Active appearance models (AAMs), audiovisual automatic speech recognition (AV-ASR), multimodal fusion, uncertainty compensation.

show abstract

mentioning

confidence: 99%

Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition

Papandreou

Katsamanis

Pitsikalis

et al. 2009

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…13 of [3], [19], [7], [5] 3 In the ML problem V −1 was used instead in order to facilitate the computations in the case of σ 2 x → 0.…”

Section: Em-sfa For Single Sequencementioning

confidence: 99%

Learning Slow Features for Behaviour Analysis

Zafeiriou

Nicolaou

Zafeiriou

et al. 2013

2013 IEEE International Conference on Computer Vision

View full text Add to dashboard Cite

A recently introduced latent feature learning technique for time varying dynamic phenomena analysis is the socalled Slow Feature Analysis (SFA). SFA is a deterministic component analysis technique for multi-dimensional sequences that by minimizing the variance of the first order time derivative approximation of the input signal finds uncorrelated projections that extract slowly-varying features ordered by their temporal consistency and constancy. In this paper, we propose a number of extensions in both the deterministic and the probabilistic SFA optimization frameworks. In particular, we derive a novel deterministic SFA algorithm that is able to identify linear projections that extract the common slowest varying features of two or more sequences. In addition, we propose an Expectation Maximization (EM) algorithm to perform inference in a probabilistic formulation of SFA and similarly extend it in order to handle two and more time varying data sequences. Moreover, we demonstrate that the probabilistic SFA (EM-SFA) algorithm that discovers the common slowest varying latent space of multiple sequences can be combined with dynamic time warping techniques for robust sequence timealignment. The proposed SFA algorithms were applied for facial behavior analysis demonstrating their usefulness and appropriateness for this task.

show abstract

“…When there are no constraints placed on the structure of the system matrices F, H, Q, and R, the estimates of the components of are (Digalakis et al, 1993;Xu and Wikle, 2004):…”

Section: Parameter Estimationmentioning

confidence: 99%

“…Thus, the following quantities are computed in the E-step and used to evaluate the sufficient statistics in Eq. (4), (Digalakis, 1993):…”

Section: The E-m Algorithm For Parameter Estimationmentioning

confidence: 99%

Using a spatio-temporal dynamic state-space model with the EM algorithm to patch gaps in daily riverflow series

Amisigo

Giesen

2005

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

Abstract.A spatio-temporal linear dynamic model has been developed for patching short gaps in daily river runoff series. The model was cast in a state-space form in which the state variable was estimated using the Kalman smoother (RTS smoother). The EM algorithm was used to concurrently estimate both parameter and missing runoff values. Application of the model to daily runoff series in the Volta Basin of West Africa showed that the model was capable of providing good estimates of missing runoff values at a gauging station from the remaining time series at the station and at spatially correlated stations in the same sub-basin.

show abstract

ML estimation of a stochastic linear system with the EM algorithm and its application to speech recognition

Cited by 207 publications

References 26 publications

Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition

Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech Recognition

Learning Slow Features for Behaviour Analysis

Using a spatio-temporal dynamic state-space model with the EM algorithm to patch gaps in daily riverflow series

Contact Info

Product

Resources

About