Maximum Likelihood Estimation of a Multi-Dimensional Log-Concave Density

Abstract

Summary. Let X 1 , ..., X n be independent and identically distributed random vectors with a (Lebesgue) density f. We first prove that, with probability 1, there is a unique log-concave maximum likelihood estimatorf n of f. The use of this estimator is attractive because, unlike kernel density estimation, the method is fully automatic, with no smoothing parameters to choose. Although the existence proof is non-constructive, we can reformulate the issue of computingf n in terms of a non-differentiable convex optimization problem, and thus combine techniques of computational geometry with Shor's r -algorithm to produce a sequence that converges tof n . An R version of the algorithm is available in the package LogConcDEAD-log-concave density estimation in arbitrary dimensions. We demonstrate that the estimator has attractive theoretical properties both when the true density is log-concave and when this model is misspecified. For the moderate or large sample sizes in our simulations,f n is shown to have smaller mean integrated squared error compared with kernel-based methods, even when we allow the use of a theoretical, optimal fixed bandwidth for the kernel estimator that would not be available in practice. We also present a real data clustering example, which shows that our methodology can be used in conjunction with the expectation-maximization algorithm to fit finite mixtures of log-concave densities.

Distinguishing Log-Concavity from Heavy Tails

“…There also exists substantial literature regarding its connections to probability theory and statistics [3,4]. Several papers concentrate on the statistical estimation of density functions assuming log-concavity [5,6]. This is due to the fact that log-concavity provides desirable statistical properties for estimators.…”

Global rates of convergence in log-concave density estimation

“…For instance, the maximum likelihood estimator of a log-concave density, first studied by Walther (2002) in the case d = 1, and by Cule, Samworth and Stewart (2010) for general d, plays a central role in all of the procedures mentioned in the previous paragraph. Dümbgen, Hüsler and Rufibach (2011) developed a fast, Active Set algorithm for computing the estimator when d = 1, and this is implemented in the R package logcondens (Rufibach and Dümbgen, 2006;.…”

“…Log-concavity therefore offers statisticians the potential of freedom from restrictive parametric (typically Gaussian) assumptions without paying a hefty price. Indeed, in recent years, researchers have sought to exploit these alluring features to propose new methodology for a wide range of statistical problems, including the detection of the presence of mixing (Walther, 2002), tail index estimation (Müller and Rufibach, 2009), clustering (Cule, Samworth and Stewart, 2010), regression , Independent Component Analysis (Samworth and Yuan, 2012) and classification (Chen and Samworth, 2013).…”

Identification of Nonparametric Simultaneous Equations Models with a Residual Index Structure

“…A strong sufficient condition is that ∂ 2 ln f (u) ∂u∂u be negative definite almost everywhere-a property of the multivariate normal and many other log-concave densities (see, e.g., Bagnoli and Bergstrom (2005) and Cule, Samworth, and Stewart (2010)). Examples of densities that violate the requirement of Corollary 2 are those that are flat (uniform) or log-linear (exponential) on an open set.…”