High-Dimensional Unsupervised Selection and Estimation of a Finite Generalized Dirichlet Mixture Model Based on Minimum Message Length

Abstract

We consider the problem of determining the structure of high-dimensional data, without prior knowledge of the number of clusters. Data are represented by a finite mixture model based on the generalized Dirichlet distribution. The generalized Dirichlet distribution has a more general covariance structure than the Dirichlet distribution and offers high flexibility and ease of use for the approximation of both symmetric and asymmetric distributions. This makes the generalized Dirichlet distribution more practical and useful. An important problem in mixture modeling is the determination of the number of clusters. Indeed, a mixture with too many or too few components may not be appropriate to approximate the true model. Here, we consider the application of the minimum message length (MML) principle to determine the number of clusters. The MML is derived so as to choose the number of clusters in the mixture model which best describes the data. A comparison with other selection criteria is performed. The validation involves synthetic data, real data clustering, and two interesting real applications: classification of web pages, and texture database summarization for efficient retrieval.

Bayesian network modeling of the consensus between experts: An application to neuron classification

“…However, the Dirichlet distribution has some constraints, e.g., its covariance matrix is strictly negative so it cannot model positive correlations between variables. Finite mixtures of generalized Dirichlet distributions [66] overcome some of these constraints. However, the generalized Dirichlet distribution has more parameters than the Dirichlet distribution, so the problem of high-dimensionality combined with few data is even more challenging.…”

MML, Hybrid Bayesian Network Graphical Models, Statistical Consistency, Invariance and Uniqueness

“…0.2.4, p. 537, col. 2] for a discussion of MML hierarchical clustering. As well as the abovementioned multinomial, Gaussian, Poisson and von Mises circular distributions [Wallace and Dowe, 1994b;1996;1997b;, this work -without sequential and spatial clustering (following in the next item) has been extended to other distributions Dowe, 2002a, 2002b;2003a;2003b;Bouguila and Ziou, 2007];…”

A Graphical Model for Content Based Image Suggestion and Feature Selection

“…p(u|z) and p(e|z) denote the likelihood of a user and context to belong respectively to the user's class z. p(r|z, c) is the probability to generate a rating for given user and image classes. We model p(v|c) using the Generalized Dirichlet distribution (GDD) [3][2] which is suitable for non Gaussian data such as images. This distribution has a more general covariance structure and provides multiple shapes.…”

“…The Fisher information of θ cl and ξ l can be computed by considering the loglikelihood of each feature taken separately [3]. After the second order derivations of this log-likelihood, we obtain:…”

“…The parameters of Beta distributions θ cl and ξ l are updated using the Fisher scoring method based on the first and second order derivatives of the MML objective [3]. In order to avoid unfavorable local optimums, we use the deterministic EM annealing [25].…”