Aggregation of Probabilistic PCA Mixtures with a Variational-Bayes Technique Over Parameters

Abstract

This paper proposes a solution to the problem of aggregating versatile probabilistic models, namely mixtures of probabilistic principal component analyzers. These models are a powerful generative form for capturing high-dimensional, non Gaussian, data. They simultaneously perform mixture adjustment and dimensionality reduction. We demonstrate how such models may be advantageously aggregated by accessing mixture parameters only, rather than original data. Aggregation is carried out through Bayesian estimation with a specific prior and an original variational scheme. Experimental results illustrate the effectiveness of the proposal.

Asynchronous gossip principal components analysis

“…This is the case of [9,11], contrarily to [6,10] which require multiple access to the X i because of their iterative optimization scheme. Temporary access to data is frequently considered in online approaches like [4] to handle stream data under short-time ergodicity and stationarity assumptions.…”

“…In [11], the authors propose an operator to aggregate Mixtures of Probabilistic PCA models (MPPCA, [12]) in a maximum-likelihood sense, without resorting to the original data used to train the models. Multiple models can thus be trained in a first phase, and aggregated in a second phase.…”

A low-cost variational-Bayes technique for merging mixtures of probabilistic principal component analyzers

“…The algorithm starts with the initialization of the model parameters. All component weights ω k are initialized to 1 K ,t h ec o m p o n e n tm e a n sµ are scattered randomly in the data domain, and the factor matrices Λ are set with random orthogonal vectors.…”

“…redundant, with regard to a model estimated on the union of the data sets. In this section, we show first how the addition of input MPPCA models can be seen as the limit representation of a virtual data set [1]. This representation is then inserted into the model described in section 3, as a substitution for an ordinary data set.…”

“…While only as h o r ta n dp r e l i m i n a r yw o r k ,c o v e r i n gm a i ni d e a sa n das i n g l ee x p e r i m e n t a ls e t t i n g (synthetic data set) could be presented in [1], we disclose herein complete technical details (both in terms of pre-requisites and novel contributions), discussion and numerous experimental results (synthetic and read data sets). Further, we focused in [1] on the contribution detailed here in section 5, while the contribution in section 4 could only be sketched in a few lines.…”