High dimensional Sparse Gaussian Graphical Mixture Model

Lotsi, Anani; Wit, Ernst

doi:10.48550/arxiv.1308.3381

Cited by 5 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For sparse Gaussian mixture model, Raftery and Dean (2006); Maugis et al (2009); Pan and Shen (2007); Maugis and Michel (2008); Städler et al (2010); Maugis and Michel (2011); Krishnamurthy (2011); Ruan et al (2011); He et al (2011); Lee and Li (2012); Lotsi and Wit (2013); Malsiner-Walli et al (2013); Azizyan et al (2013); Gaiffas and Michel (2014) study the problems of clustering and feature selection, but mostly either lack efficient algorithms to attain the proposed estimators, or do not have finite-sample guarantees. Verzelen and Arias-Castro (2017); Azizyan et al (2015) establish efficient algorithms for detection, feature selection, and clustering with finite-sample guarantees.…”

Section: Introductionmentioning

confidence: 99%

Curse of Heterogeneity: Computational Barriers in Sparse Mixture Models and Phase Retrieval

Fan¹,

Liu²,

Wang³

et al. 2018

Preprint

View full text Add to dashboard Cite

We study the fundamental tradeoffs between statistical accuracy and computational tractability in the analysis of high dimensional heterogeneous data. As examples, we study sparse Gaussian mixture model, mixture of sparse linear regressions, and sparse phase retrieval model. For these models, we exploit an oracle-based computational model to establish conjecture-free computationally feasible minimax lower bounds, which quantify the minimum signal strength required for the existence of any algorithm that is both computationally tractable and statistically accurate. Our analysis shows that there exist significant gaps between computationally feasible minimax risks and classical ones. These gaps quantify the statistical price we must pay to achieve computational tractability in the presence of data heterogeneity. Our results cover the problems of detection, estimation, support recovery, and clustering, and moreover, resolve several conjectures of Azizyan et al. (2013Azizyan et al. ( , 2015; Verzelen and Arias-Castro (2017); Cai et al. (2016). Interestingly, our results reveal a new but counter-intuitive phenomenon in heterogeneous data analysis that more data might lead to less computation complexity.

show abstract

Section: Introductionmentioning

confidence: 99%

Curse of Heterogeneity: Computational Barriers in Sparse Mixture Models and Phase Retrieval

Fan¹,

Liu²,

Wang³

et al. 2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Rodríguez et al (2011) and Talluri et al (2014) develop a Bayesian framework for estimating infinite mixtures of sparse Gaussian graphical models where different prior distributions on the inverse covariance are employed. Krishnamurthy (2011), Lotsi and Wit (2013), Gao et al (2016), and Lee and Xue (2017) present methods for estimating mixture models with sparse precision matrices via penalized likelihood estimation and lasso-type penalty functions. Compared to these approaches, in our proposed framework we parameterize the mixture of Gaussians directly in terms of the component covariance matrices.…”

Section: Discussionmentioning

confidence: 99%

Model-based clustering with sparse covariance matrices

2018

View full text Add to dashboard Cite

Finite Gaussian mixture models are widely used for model-based clustering of continuous data. Nevertheless, since the number of model parameters scales quadratically with the number of variables, these models can be easily over-parameterized. For this reason, parsimonious models have been developed via covariance matrix decompositions or assuming local independence. However, these remedies do not allow for direct estimation of sparse covariance matrices nor do they take into account that the structure of association among the variables can vary from one cluster to the other. To this end, we introduce mixtures of Gaussian covariance graph models for model-based clustering with sparse covariance matrices. A penalized likelihood approach is employed for estimation and a general penalty term on the graph configurations can be used to induce different levels of sparsity and incorporate prior knowledge. Model estimation is carried out using a structural-EM algorithm for parameters and graph structure estimation, where two alternative strategies based on a genetic algorithm and an efficient stepwise search are proposed for inference. With this approach, sparse component covariance matrices are directly obtained. The framework results in a parsimonious model-based clustering of the data via a flexible model for the within-group joint distribution of the variables. Extensive simulated data experiments and application to illustrative datasets show that the method attains good classification performance and model quality.

show abstract

“…Lastly, state-of-the-art prediction methods on time-series [22,15,17,25] commonly assume the relations among the past values of the variables and the present values of each variable to be constant in time. An idea similar to TAGM was proposed with Gaussian Mixture Models (GMMs) [7] where they combined GMM with GGMs [18]. The use of GMMs though would not allow to explicitly consider sequentiality and it is therefore not suited for the analysis of time-series.…”

Section: Related Workmentioning

confidence: 99%

“…Infer temporal-dependent conditional dependencies among variables: TAGM relaxed the chunk assumption common to the inference methods available in literature [13,24] thus inferring a time-varying network that adapts at each observation. Note that TAGM assumes a sequentiality of the states, while in [18,7] the authors combined GGMs with Gaussian Mixture Models. This approach is more prone in clustering the points in classes which can be seen as an unsupervised extension of the Joint Graphical Lasso [6].…”

Section: Introductionmentioning

confidence: 99%

Time Adaptive Gaussian Model

Ciech,

Tozzo

2021

Preprint

View full text Add to dashboard Cite

Multivariate time series analysis is becoming an integral part of data analysis pipelines. Understanding the individual time point connections between covariates as well as how these connections change in time is non-trivial. To this aim, we propose a novel method that leverages on Hidden Markov Models and Gaussian Graphical Models -Time Adaptive Gaussian Model (TAGM). Our model is a generalization of state-of-the-art methods for the inference of temporal graphical models, its formulation leverages on both aspects of these models providing better results than current methods. In particular,it performs pattern recognition by clustering data points in time; and, it finds probabilistic (and possibly causal) relationships among the observed variables. Compared to current methods for temporal network inference, it reduces the basic assumptions while still showing good inference performances.

show abstract

High dimensional Sparse Gaussian Graphical Mixture Model

Cited by 5 publications

References 0 publications

Curse of Heterogeneity: Computational Barriers in Sparse Mixture Models and Phase Retrieval

Curse of Heterogeneity: Computational Barriers in Sparse Mixture Models and Phase Retrieval

Model-based clustering with sparse covariance matrices

Time Adaptive Gaussian Model

Contact Info

Product

Resources

About