Context-GMM: Incremental learning of sparse priors for Gaussian mixture regression

Abstract

Abstract-Gaussian Mixture Models have been widely used in robotic control and in sensory anticipation applications. A mixture model is learnt from demonstrations and later used to infer the most likely control signals, or is also used as a forward model to predict the change in sensory signals over time. However, such models often are too big to be tractable in real-time applications. In this paper we introduce the Context-GMM, a method to learn sparse priors over the mixture components. Such priors are stable over large amounts of time and provide a way of selecting very small subsets of mixture components without significant loss in accuracy and with huge computational savings.

A distributed incremental update scheme for probability distribution of wind power forecast error

“…For the creating step, if (22) does not hold, then a new Gaussian component should be created to accommodate the new information carried by the new data. The θ new j,m of this new component are initialized by ( 27)- (30), where ∆ ini is a preset parameter.…”

“…The first step is to decide whether a piece of new data belongs to an existing Gaussian component in the GMM. Different criteria are utilized: the likelihood value of a component when substituting the latest data in it [30], the distance between the latest data and the expected value of the components [29], or the Mahalanobis distance between the latest data and the components [28], [31], [32]. If the judgment passed, i.e., the new data belongs to an existing Gaussian component, then the parameters of the GMM will be updated based on the new data in the second step.…”

“…If the judgment passed, i.e., the new data belongs to an existing Gaussian component, then the parameters of the GMM will be updated based on the new data in the second step. The Robbins-Monro stochastic approximation method is the most common method to estimate the updated parameters of the GMM [30] and is improved in [31], [32] by introducing the inverse of the covariance to avoid the need for the inverse calculation in each update. The filter-based algorithms, i.e., the causal low-pass filter [29] and self-adapted recursive filter [28], are also used to update the parameters.…”

“…The filter-based algorithms, i.e., the causal low-pass filter [29] and self-adapted recursive filter [28], are also used to update the parameters. If the judgment is not passed, then a new Gaussian component is created [30]- [32], or the less contributing component is replaced [28] in the third step with a preset covariance and a mean vector equal to the new data.…”