On the smoothing of multinomial estimates using Liouville mixture models and applications

Abstract

There has been major progress in recent years in statistical model-based pattern recognition, data mining and knowledge discovery. In particular, generative models are widely used and are very reliable in terms of overall performance. Success of these models hinges on their ability to construct a representation which captures the underlying statistical distribution of data. In this article we focus on count data modeling. Indeed, this kind of data is naturally generated in many contexts and in different application domains. Usually, models based on the multinomial assumption are used in this case which may have several shortcomings especially in the case of high-dimensional sparse data. We propose then a principled approach to smooth multinomials using a mixture of Beta-Liouville distributions which is learned to reflect and model prior beliefs about multinomial parameters. Via both theoretical interpretations and experimental validations, we argue that the proposed smoothing model is general and flexible enough to allow accurate representation of count data.

Revisiting the Optimal Probability Estimator from Small Samples for Data Mining

“…The strength of the push is proportional to the value of α. The process of Bayesian probability estimation is sometimes called smoothing or flattening (Flach, 2012;Fienberg and Holland, 1972;Bouguila, 2013). The main role of smoothing is to push the estimated probability away from the extreme value obtained with relative frequency (1) towards some sort of average value (1/2 for Laplace's and Piegat's formulae, p a for the m-estimate).…”

Exploiting optical signal analysis for autonomous communications

“…Differently from the k-means, a hard-clustering algorithm, GMMs [MMA20] are softclustering probabilistic models, that describe each cluster in a dataset by a Gaussian distribution characterized by its mean and variance. Notice that, when we are working two independent random variables, GMMs output bivariate Gaussian distributions characterized by its mean, variance, and covariance.…”

“…In addition, by forcing the constellation points to be modeled as bivariate Gaussian distributions and initializing the GMM fitting with the expected constellation 50 PhD Thesis -Diogo Gonçalo Sequeira centroids, the obtained features are strongly conditioned. For this reason, we compute the loglikelihood value ℒ as a goodness-of-fit metric to estimate GMM fitting accuracy [MMA20].…”

“…Fig.6-3 left an input sample X from a 16QAM optical signal, i.e., m=16. We apply GMM[MMA20] to characterize a given OC sample as multiple bivariate Gaussian distributions, i.e., one distribution for each constellation point. Then, for 16QAM signals, we obtain 16 different distributions.Fig.…”