Adaptive Mixtures of Local Experts

Jacobs, Robert A.; Jordan, Michael I.; Nowlan, Steven J.; Hinton, Geoffrey E.

doi:10.1162/neco.1991.3.1.79

Cited by 3,460 publications

(1,968 citation statements)

References 1 publication

Supporting

Mentioning

1,841

Contrasting

Unclassified

Order By: Relevance

“…S1 in Supplementary Material). This architecture was constructed in a simulation study of object manipulation (Gomi and Kawato 1993) based on a mixture-of-experts model (Jacobs et al 1991). The mixture-of-experts model involves expert modules, which are equivalent to internal inverse models, and a gating module.…”

Section: Simulationsmentioning

confidence: 99%

Explicit contextual information selectively contributes to predictive switching of internal models

Imamizu¹,

Sugimoto²,

Osu

et al. 2007

Exp Brain Res

View full text Add to dashboard Cite

Many evidences suggest that the central nervous system (CNS) acquires and switches internal models for adaptive control in various environments. However, little is known about the neural mechanisms responsible for the switching. A recent computational model for simultaneous learning and switching of internal models proposes two separate switching mechanisms: a predictive mechanism purely based on contextual information and a postdictive mechanism based on the difference between actual and predicted sensorimotor feedbacks. This model can switch internal models solely based on contextual information in a predictive fashion immediately after alteration of the environment. Here we show that when subjects simultaneously adapted to alternating blocks of opposing visuomotor rotations, explicit contextual information about the rotations improved the initial performance at block alternations and asymptotic levels of performance within each block but not readaptation speeds. Our simulations using separate switching mechanisms duplicated these effects of contextual information on subject performance and suggest that improvement of initial performance was caused by improved accuracy of the predictive switch while adaptation speed corresponds to a switch dependent on sensorimotor feedback. Simulations also suggested that a slow change in output signals from the switching mechanisms causes contamination of motor commands from an internal model used in the previous context (anterograde interference) and partial destruction of internal models (retrograde interference). Explicit contextual information prevents destruction and assists memory retention by improving the changes in output signals. Thus, the asymptotic levels of performance improved.

show abstract

Section: Simulationsmentioning

confidence: 99%

Explicit contextual information selectively contributes to predictive switching of internal models

Imamizu¹,

Sugimoto²,

Osu

et al. 2007

Exp Brain Res

View full text Add to dashboard Cite

show abstract

“…Mixture of experts (Jacobs et al, 1991;Jordan and Jacobs, 1994) are used in a variety of contexts including regression, classification and clustering. Here we consider the MoE framework for fitting (non-linear) regression functions and clustering of univariate continuous data .…”

Section: Mixture Of Experts For Continuous Datamentioning

confidence: 99%

“…Mixture of experts (MoE) introduced by Jacobs et al (1991) are widely studied in statistics and machine learning. They consist in a fully conditional mixture model where both the mixing proportions, known as the gating functions, and the component densities, known as the experts, are conditional on some input covariates.…”

Section: Introductionmentioning

confidence: 99%

Robust mixture of experts modeling using thetdistribution

Chamroukhi¹

2016

Neural Networks

View full text Add to dashboard Cite

Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification, and clustering. For regression and cluster analyses of continuous data, MoE usually use normal experts following the Gaussian distribution. However, for a set of data containing a group or groups of observations with heavy tails or atypical observations, the use of normal experts is unsuitable and can unduly affect the fit of the MoE model. We introduce a robust MoE modeling using the t distribution. The proposed t MoE (TMoE) deals with these issues regarding heavy-tailed and noisy data. We develop a dedicated expectation-maximization (EM) algorithm to estimate the parameters of the proposed model by monotonically maximizing the observed data log-likelihood. We describe how the presented model can be used in prediction and in model-based clustering of regression data. The proposed model is validated on numerical experiments carried out on simulated data, which show the effectiveness and the robustness of the proposed model in terms of modeling non-linear regression functions as well as in model-based clustering. Then, it is applied to the real-world data of tone perception for musical data analysis, and the one of temperature anomalies for the analysis of climate change data. The obtained results show the usefulness of the TMoE model for practical applications.

show abstract

“…The algorithms for training the mixture of autoencoders are related to a formulation of the adaptive mixture model (Jacobs & Jordan, 1993;Jacobs, Jordan, Nowlan & Hinton, 1991;Jordan & Jacobs, 1994). The adaptive mixture model can be regarded as a supervised version of the mixture of autoencoders model.…”

Section: Unsupervised Training Based On Maximum Likelihood Estimationmentioning

confidence: 99%

Unsupervised visual learning of three-dimensional objects using a modular network architecture

1999

View full text Add to dashboard Cite

This paper presents a modular network architecture that learns to cluster multiple views of multiple three-dimensional (3D) objects. The proposed network model is based on a mixture of non-linear autoencoders, which compete to encode multiple views of each 3D object. The main advantage of using a mixture of autoencoders is that it can capture multiple non-linear sub-spaces, rather than multiple centers for describing complex shapes of the view distributions. The unsupervised training algorithm is formulated within a maximum-likelihood estimation framework. The performance of the modular network model is evaluated through experiments using synthetic 3D wire-frame objects and gray-level images of real 3D objects. It is shown that the performance of the modular network model is superior to the performance of the conventional clustering algorithms, such as the K-means algorithm and the Gaussian mixture model. ᭧

show abstract

Adaptive Mixtures of Local Experts

Cited by 3,460 publications

References 1 publication

Explicit contextual information selectively contributes to predictive switching of internal models

Explicit contextual information selectively contributes to predictive switching of internal models

Robust mixture of experts modeling using thetdistribution

Unsupervised visual learning of three-dimensional objects using a modular network architecture

Contact Info

Product

Resources

About