Application of Clustering in the Non-Parametric Estimation of Distribution Density

Abstract

This paper discusses a multimodal density function estimation problem of a random vector. A comparative accuracy analysis of some popular non-parametric estimators is made by using the Monte-Carlo method. The paper demonstrates that the estimation quality increases significantly if the sample is clustered (i.e., the multimodal density function is approximated by a mixture of unimodal densities), and later on, the density estimation methods are applied separately to each cluster. In this paper, the sample is clustered using the Gaussian distribution mixture model and the EM algorithm. The highest efficiency in the analysed cases was reached by using the iterative procedure proposed by Friedman for estimating a density component corresponding to each cluster after the primary sample clustering mentioned. The Friedman procedure is based on both the projection pursuit of multivariate observations and transformation of the univariate projections into the standard Gaussian random values (using the density function estimates of these projections).

On clustering procedures and nonparametric mixture estimation

“…Many authors have already proposed to carry out a preliminary clustering step to improve density estimates in mixture models. Ruzgas et al (2006) conduct a comprehensive simulation study to conclude that a preliminary clustering using the EM algorithm allows to some extent to improve performances of some density estimates (see also Jeon and Landgrebe (1994)). However, to our knowledge, no work has been devoted so far to measure the effects of the clustering algorithm on the resulting estimates of the distribution functions f i .…”

Accuracy of Nonparametric Density Estimation for Univariate Gaussian Mixture Models: A Comparative Study

“…Although previous numerical simulations [32] have shown that initial data clustering is beneficial for evaluating multimodal distribution density, the question on the optimal clustering method is still open. A comparative study in [32] showed that probabilistic clustering methods are more robust compared to geometric clustering methods (e.g., k -means, hierarchical, etc.) applied in conjunction with the transformation of mixture components to spherical [34].…”

“…Academia and industry focus on the development of innovative density estimation procedures (see in [25,30]). In some cases the accuracy of the assessment can be significantly increased (according to [32]), if the observations are clustered at first (i.e., treating the multimodal density as a mixture of unimodal densities), and the popular nonparametric estimators are applied to each cluster separately.…”

“…The present work extended the pilot research presented in [32]. The following new research areas are investigated: 1) new density estimation methods based on inverse formula are formulated, 2) in order to compare the robustness of estimators the wide set of distributions proposed by Marron and Wand [26] is used to study densities, 3) in order to obtain results with a reasonably high level of confidence a relatively high (100000) number of independent samples have been generated.…”

“…In this work, after the clustering pre-processing step, the mixture components were identified using: 1) a kernel estimator with the adaptively selected smoothing width, 2) Friedmans proposed algorithm based on the projection pursuit, 3) Kooperberg and Stone log-spline estimator based on the approximation of logarithm of distribution density by the sum of cubic B-splines, 4) Fix and Hodges k -nearest neighbor density estimator, and 5) the proposed inversion formula modification for density estimator, where the structure is given in (3.6). A pilot comparative study of several non-parametric estimators accuracy [32] showed that the Friedman procedure is more robust in the majority of examined Gaussian multivariate mixtures cases where the components can be separated. The kernel estimator is more accurate even with a small sample size.…”