An ensemble learning approach to independent component analysis

Choudrey, Rizwan; Penny, William D.; Roberts, Stephen

doi:10.1109/nnsp.2000.889436

Cited by 20 publications

(15 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this section we analyse theoretically how the choice of the form of the posterior approximation q(S) of the sources affects the solution which optimises the cost function (4).…”

Section: Effect Of Posterior Approximation: Theorymentioning

confidence: 99%

See 1 more Smart Citation

On the Effect of the Form of the Posterior Approximation in Variational Learning of ICA Models

Ilin

Valpola²

2005

Neural Process Lett

View full text Add to dashboard Cite

We show that the choice of posterior approximation of sources affects the solution found in Bayesian variational learning of linear independent component analysis models. Assuming the sources to be independent a posteriori favours a solution which has an orthogonal mixing matrix. A linear dynamic model which uses second-order statistics is considered but the analysis extends to nonlinear mixtures and non-Gaussian source models as well.

show abstract

“…In this section we analyse theoretically how the choice of the form of the posterior approximation q(S) of the sources affects the solution which optimises the cost function (4).…”

Section: Effect Of Posterior Approximation: Theorymentioning

confidence: 99%

“…Recently several methods for variational Bayesian learning of linear ICA models and their extensions have been reported in the literature [1,2,3,4,5,6,7,8]. The basic idea in these approaches is to approximate the true posterior probability density of the unknown variables by a function which has a restricted form.…”

Section: Introductionmentioning

confidence: 99%

On the Effect of the Form of the Posterior Approximation in Variational Learning of ICA Models

Ilin

Valpola²

2005

Neural Process Lett

View full text Add to dashboard Cite

show abstract

“…The prior over the precisions is a Gamma distribution (9) the prior over the means is a univariate normal (10) and the prior over the weights is a zero-mean Gaussian with an isotropic covariance having precision (11) where…”

Section: A Model Priorsmentioning

confidence: 99%

“…Although the Bayesian methodology has a long history, the use of VB is relatively new; the key idea of VB is to find a tractable approximation to the true posterior density that minimizes the Kullback-Leibler (KL) divergence [8]. Notable recent applications are to principal component analysis [9] and independent component analysis [10]. We have also published short conference papers summarizing some key results for standard autoregressive models [11] and non-Gaussian AR models [12].…”

Section: Introductionmentioning

confidence: 99%

Variational Bayes for generalized autoregressive models

Roberts

Penny

2002

IEEE Trans. Signal Process.

Self Cite

136

View full text Add to dashboard Cite

Abstract-We describe a variational Bayes (VB) learning algorithm for generalized autoregressive (GAR) models. The noise is modeled as a mixture of Gaussians rather than the usual single Gaussian. This allows different data points to be associated with different noise levels and effectively provides robust estimation of AR coefficients. The VB framework is used to prevent overfitting and provides model-order selection criteria both for AR order and noise model order. We show that for the special case of Gaussian noise and uninformative priors on the noise and weight precisions, the VB framework reduces to the Bayesian evidence framework. The algorithm is applied to synthetic and real data with encouraging results.Index Terms-Bayesian inference, generalized autoregressive models, model order selection, robust estimation.

show abstract

“…It has been applied to ICA and a wide variety of other models (see e.g. Hinton and van Camp, 1993;Barber and Bishop, 1998;Attias, 1999;Miskin and MacKay, 2000;Ghahramani and Hinton, 2000;Choudrey et al, 2000;Chan et al, 2001;Valpola and Karhunen, 2002). An example of applying a variational technique other than ensemble learning to linear ICA has been given by Girolami (2001).…”

Section: Variational Bayesian Learningmentioning

confidence: 99%

Hierarchical models of variance sources

2004

View full text Add to dashboard Cite

In many models, variances are assumed to be constant although this assumption is often unrealistic in practice. Joint modelling of means and variances is difficult in many learning approaches, because it can lead into infinite probability densities. We show that a Bayesian variational technique which is sensitive to probability mass instead of density is able to jointly model both variances and means. We consider a model structure where a Gaussian variable, called variance node, controls the variance of another Gaussian variable. Variance nodes make it possible to build hierarchical models for both variances and means. We report experiments with artificial data which demonstrate the ability of the learning algorithm to find variance sources explaining and characterizing well the variances in the multidimensional data. Experiments with biomedical MEG data show that variance sources are present in real-world signals.

show abstract

An ensemble learning approach to independent component analysis

Cited by 20 publications

References 13 publications

On the Effect of the Form of the Posterior Approximation in Variational Learning of ICA Models

On the Effect of the Form of the Posterior Approximation in Variational Learning of ICA Models

Variational Bayes for generalized autoregressive models

Hierarchical models of variance sources

Contact Info

Product

Resources

About