Nonparametric Roughness Penalties for Probability Densities

Abstract

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.. Biometrika Trust is collaborating with JSTOR to digitize, preserve and extend access to Biometrika. SUMMARY Given a number of observations xl, ..., xN, a nonparametric method is suggested for estimating the entire probability density curve. The method is to subtract a roughness penalty from the log likelihood, where the roughness penalty is a certain functional of the assumed density function f. Those used are linear combinations off y'2dx andf y"2dx, where y = If. The method appears to be consistent under wide conditions, although consistent methods can be rough. Numerical examples are given and show that for certain values of the coefficients in this linear expression the density function turns out to be very smooth even when N is small. Multivariate extensions are proposed, including one to distributions having some continuous and some discrete components, but numerical examples of these have not been tried. Some of the techniques are borrowed from quantum mechanics and tensor calculus.

Nonparametric Estimation of Information-Based Measures of Statistical Dispersion

“…We found that the maximum penalized likelihood (MPL) method of Good and Gaskins [26] for probability density estimation offers a possible solution. The idea is to represent the whole probability density function using a suitable orthogonal base of functions (Hermite functions), and the "best" estimate of f (t) is obtained by solving a set of equations for the base coefficients given the sample…”

“…and Φ is the roughness penalty proposed by [26], controlling the smoothness of the density of X (and hence the smoothness of f (t)),…”

“…In accordance with [26], the algorithm is stopped when λ does not change within desired precision in subsequent iterations. After the score-maximizing c i 's are obtained, and thus the density of X is estimated, the estimates of dispersion coefficients in (2) and (4) are calculated from the following entropy and Fisher information estimators.…”

“…The larger bases, r = 50 or r = 100, were examined too, with negligible differences in the estimation for the selected models. The values of the parameters for (15) were chosen as α = 3 and β = 4, in accordance with the suggestion ( [26], Appendix A).…”

Penalized Likelihood for General Semi-Parametric Regression Models

“…At least there will be unidentifiability of parameters; possibly a 'Dirac catastrophe'. In the context of density estimation, Good and Gaskins (1971) -2-.…”