Clustering via finite nonparametric ICA mixture models

Abstract

We propose a novel extension of nonparametric multivariate finite mixture models by dropping the standard conditional independence assumption and incorporating the independent component analysis (ICA) structure instead. This innovation extends nonparametric mixture model estimation methods to situations in which conditional independence, a necessary assumption for the unique identifiability of the parameters in such models, is clearly violated. We formulate an objective function in terms of penalized smoothed Kullback-Leibler distance and introduce the nonlinear smoothed majorizationminimization independent component analysis (NSMM-ICA) algorithm for optimizing this function and estimating the model parameters. Our algorithm does not require any labeled observations a priori; it may be used for fully unsupervised clustering problems in a multivariate setting. We have implemented a practical version of this algorithm, which utilizes the FastICA algorithm, in the R package icamix. We illustrate this new methodology using several applications in unsupervised learning and image processing.

Simultaneous semi-parametric estimation of clustering and regression

“…The semi-parametric approach has been presented by assuming that the components are products of univariate densities. However, the proposed approach can also be used by considering location scale symmetric distributions (Hunter et al, 2007) or by incorporating an independent component analysis structure (Zhu and Hunter, 2019). Moreover, we can easily relax the assumption that (X , Z ) is independent of U .…”

Online nonparametric Bayesian analysis of parsimonious Gaussian mixture models and scenes clustering

“…The usual practice is to presuppose a K value, but this often leads to overfitting or underfitting unless the researchers have sufficient empirical knowledge of the data sources and are able to make the right choices. To deal with these troubles, the Bayesian nonparametric mixture model is gradually emerging [13,14], which has the Dirichlet process as its stepping stone [15]. By providing the model a special prerequisite, the number of components is not fixed in advance, but the model is assumed to have infinite hybrid components which means it has infinite parameters.…”

Heart Disease Diagnosis via Nonparametric Mixture Models

“…In this paper we focus on nonparametric mixture models on multivariate mixed count data for heart disease using Gaussian kernel density estimators. Although most nonparametric mixture models have been built under the assumption of identically independent components for identifiability, latent variable models with Gaussian kernels mixtures distributions can be built from observed data of mixed type which can be ordinary, binary, continuous and with component independence etc as explained in [3,4], respectively.…”