Bounds for the number of modes of the simplest Gaussian mixture

Aprausheva, N. N.; Mollaverdi, Naser; Sorokin, S. V.

doi:10.1134/s1054661806040158

Cited by 2 publications

(3 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For each value of k, we make 1000 trial experiments to gather statistics and use s = Notice that the quality of the approximations of DJ depend on the number of modes of the GMMs. However, calculating the number of modes is difficult [43,70], even for simple cases [71,72].…”

Section: Experiments: Jeffreys Divergence Between Mixturesmentioning

confidence: 99%

Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions

Nielsen

2021

Entropy

View full text Add to dashboard Cite

The Jeffreys divergence is a renown arithmetic symmetrization of the oriented Kullback–Leibler divergence broadly used in information sciences. Since the Jeffreys divergence between Gaussian mixture models is not available in closed-form, various techniques with advantages and disadvantages have been proposed in the literature to either estimate, approximate, or lower and upper bound this divergence. In this paper, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate Gaussian mixtures with arbitrary number of components. Our heuristic relies on converting the mixtures into pairs of dually parameterized probability densities belonging to an exponential-polynomial family. To measure with a closed-form formula the goodness of fit between a Gaussian mixture and an exponential-polynomial density approximating it, we generalize the Hyvärinen divergence to α-Hyvärinen divergences. In particular, the 2-Hyvärinen divergence allows us to perform model selection by choosing the order of the exponential-polynomial densities used to approximate the mixtures. We experimentally demonstrate that our heuristic to approximate the Jeffreys divergence between mixtures improves over the computational time of stochastic Monte Carlo estimations by several orders of magnitude while approximating the Jeffreys divergence reasonably well, especially when the mixtures have a very small number of modes.

show abstract

Section: Experiments: Jeffreys Divergence Between Mixturesmentioning

confidence: 99%

Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions

Nielsen

2021

Entropy

View full text Add to dashboard Cite

show abstract

“…In statistics, those matrices A are called moment matrices and well-studied [51,50,83]. The variance Var[X] of a random variable X can be expressed as the determinant of the order-2 moment matrix 5 :…”

Section: Integral-based Score Matching Estimator (Sme)mentioning

confidence: 99%

“…Notice that the quality of the approximations depend on the number of modes of the GMMs. This is a difficult problem to calculate [20,3] even for simple cases [5,6].…”

Section: Experiments: Jeffreys Divergence Between Mixturesmentioning

confidence: 99%

Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities

Nielsen

2021

Preprint

View full text Add to dashboard Cite

The Jeffreys divergence is a renown symmetrization of the statistical Kullback-Leibler divergence which is often used in statistics, machine learning, signal processing, and information sciences in general. Since the Jeffreys divergence between the ubiquitous Gaussian Mixture Models are not available in closed-form, many techniques with various pros and cons have been proposed in the literature to either (i) estimate, (ii) approximate, or (iii) lower and/or upper bound this divergence. In this work, we propose a simple yet fast heuristic to approximate the Jeffreys divergence between two univariate GMMs of arbitrary number of components. The heuristic relies on converting GMMs into pairs of dually parameterized probability densities belonging to exponential families. In particular, we consider Exponential-Polynomial Densities, and design a goodness-of-fit criterion to measure the dissimilarity between a GMM and a EPD which is a generalization of the Hyvärinen divergence. This criterion allows one to select the orders of the EPDs to approximate the GMMs. We demonstrate experimentally that the computational time of our heuristic improves over the stochastic Monte Carlo estimation baseline by several orders of magnitude while approximating reasonably well the Jeffreys divergence, specially when the univariate mixtures have a small number of modes.

show abstract

Bounds for the number of modes of the simplest Gaussian mixture

Cited by 2 publications

References 2 publications

Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions

Fast Approximations of the Jeffreys Divergence between Univariate Gaussian Mixtures via Mixture Conversions to Exponential-Polynomial Distributions

Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities

Contact Info

Product

Resources

About