EM algorithms for ML factor analysis

Rubin, Donald B.; Thayer, Dorothy T.

doi:10.1007/bf02293851

Cited by 469 publications

(350 citation statements)

References 2 publications

Supporting

Mentioning

349

Contrasting

Order By: Relevance

“…It can be used as a structured covariance matrix scheme, as an alternative to using full covariance matrices [14], [15], [17]. The generative model for the shared FA [14] extension of FA can be written as…”

Section: B Relationship To Factor Analysismentioning

confidence: 99%

“…By viewing VTS and JUD in terms of FA-related approaches it is possible to use the EM-related FA training approaches [14], [15] to perform adaptive training. This is the approach used in [7], though not described in terms of an FA related process.…”

Section: B Relationship To Factor Analysismentioning

confidence: 99%

“…In the next section model-based, predictive compensation approaches, using JUD and VTS, are briefly reviewed, as well as illustrating the close relationship between these approaches and Factor Analysis (FA) style models [14], [15]. Schemes for ML adaptive training are then discussed.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Discriminative adaptive training with VTS and JUD

Flego

Gales

2009

2009 IEEE Workshop on Automatic Speech Recognition &Amp; Understanding

View full text Add to dashboard Cite

Abstract-Adaptive training is a powerful approach for building speech recognition systems on non-homogeneous training data. Recently approaches based on predictive model-based compensation schemes, such as Joint Uncertainty Decoding (JUD) and Vector Taylor Series (VTS), have been proposed. This paper reviews these model-based compensation schemes and relates them to factor-analysis style systems. Forms of Maximum Likelihood (ML) adaptive training with these approaches are described, based on both second-order optimisation schemes and Expectation Maximisation (EM). However, discriminative training is used in many state-of-the-art speech recognition. Hence, this paper proposes discriminative adaptive training with predictive model-compensation approaches for noise robust speech recognition. This training approach is applied to both JUD and VTS compensation with minimum phone error training. A large scale multi-environment training configuration is used and the systems evaluated on a range of in-car collected data tasks.I. INTRODUCTION Speech recognition in noisy environments is a difficult task. There have been a range of solutions proposed to this problem [1], [2], [3]. One approach that has been successfully applied in a range of tasks is model-based compensation. These schemes include Vector Taylor Series (VTS), [1], and Joint Uncertainty Decoding (JUD), [4]. Here a "clean" acoustic model is adapted to be representative of a model trained in the target acoustic condition. To apply these approaches it is necessary to estimate a background noise model and, using a mismatch function that represents the impact of noise on the clean speech, combine this with the clean model parameters. Schemes have previously been proposed that allow the noise model parameters to be robustly trained using Maximum Likelihood (ML) estimation [3], [5] and will not be discussed in this paper. However, it is not always possible to directly train clean model parameters. Current state-of-the-art speech recognition systems are trained on large amounts of speech data, typically collected in a range of acoustic conditions. It is also expected that collecting data in conditions related to the actual target application conditions should improve performance.To address this problem adaptive training has been proposed [5], [6], [7], [8]. These schemes make use of adaptation/compensation transformations during training. The aim is to train a canonical model removing any dependence on speaker or noise condition. Originally proposed to account for the differences between speakers in the training data [6], these schemes have more recently been applied to situations where there are large variations in the background noise [5], [7]. Model-based compensation schemes have been used [5],

show abstract

Section: B Relationship To Factor Analysismentioning

confidence: 99%

Section: B Relationship To Factor Analysismentioning

confidence: 99%

See 1 more Smart Citation

Discriminative adaptive training with VTS and JUD

Flego

Gales

2009

2009 IEEE Workshop on Automatic Speech Recognition &Amp; Understanding

View full text Add to dashboard Cite

show abstract

“…where Sk is the sample mean corrected SSCP matrix, Ck is defined in (4) Rubin and Thayer (1982) in the standard factor analysis context, has the definite advantage of simplicity of the calculation involved due to the linear (tri-linear) nature of the complete data likelihood. Also, the algorithm is flexible enough to incorporate the Bayesian approach shown in Mayekawa (1985).…”

Section: The Maximum Likelihood Estimation By the Em Algorithmmentioning

confidence: 99%

Maximum Likelihood Solution to the Parafac Model

Mayekawa

1987

Behaviormetrika

View full text Add to dashboard Cite

“…Although Bentler's approach avoids discarding data, it is not fully efficient. Jamshidian (1997) gave an extension of the Expectation-Maximization (EM) algorithm of Rubin and Thayer (1982) to obtain for the confirmatory factor analysis (CFA) model when data are incomplete. Generalization of his algorithm, however, to more complex mean and covariance structure models is not trivial.…”

Section: Introductionmentioning

confidence: 99%

ML Estimation of Mean and Covariance Structures with Missing Data Using Complete Data Routines

Jamshidian¹,

Bentler²

1999

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

We consider maximum likelihood (ML) estimation of mean and covariance structure models when data are missing. Expectation maximization (EM), generalized expectation maximization (GEM), Fletcher-Powell, and Fisherscoring algorithms are described for parameter estimation. It is shown how the machinery within a software that handles the complete data problem can be utilized to implement each algorithm. A numerical differentiation method for obtaining the observed information matrix and the standard errors is given. This method too uses the complete data program machinery. The likelihood ratio test is discussed for testing hypotheses. Three examples are used to compare the cost of the four algorithms mentioned above, as well as to illustrate the standard error estimation and the test of hypothesis considered.The sensitivity of the ML estimates as well as the mean imputed and listwise deletion estimates to missing data mechanisms is investigated using three artificial data sets that are missing completely at random (MCAR), missing at random (MAR), and neither MCAR nor MAR.

show abstract

EM algorithms for ML factor analysis

Abstract: factor analysis, EM algorithms, maximum likelihood,

Cited by 469 publications

References 2 publications

Discriminative adaptive training with VTS and JUD

Discriminative adaptive training with VTS and JUD

Maximum Likelihood Solution to the Parafac Model

ML Estimation of Mean and Covariance Structures with Missing Data Using Complete Data Routines

Contact Info

Product

Resources

About