Correntropy: Implications of nonGaussianity for the moment expansion and deconvolution

Abstract

The recently introduced correntropy function is an interesting and useful similarity measure between two random variables which has found myriad applications in signal processing. A series expansion for correntropy in terms of higher-order moments of the difference between the two random variables has been used to try to explain its statistical properties for uses such as deconvolution. We examine the existence and form of this expansion, showing that it may be divergent, e.g., when the difference has the Laplace distribution, and give sufficient conditions for its existence for differently characterized sub-Gaussian distributions. The contribution of the higher-order moments can be quite surprising, depending on the size of the Gaussian kernel in the definition of the correntropy. In the blind deconvolution setting we demonstrate that statistical exchangeability explains the existence of sub-optimal minima in the correntropy cost surface and show how the positions of these minima are controlled by the size of the Gaussian kernel.

A Comparison Between Kernel-based Adaptive Filters Including the Epanechnikov Function

“…where σ is the kernel width, X and Y are two arbitrary random variables, E[•] is the expectation operator, κ(•) is a symmetric positive definite kernel function. Correntropy is calculated using the Gaussian kernel in most works found in the literature [15], [17] :…”

An Introduction to Information Theoretic Learning, Part II: Applications

“…A careful analysis of (4) was considered in [4], where the authors show that the series may diverge depending on the distribution of the signal being considered. However, for shorter-tailed distributions such as the uniform, it is also possible to derive certain conditions for which the series converges [4]. Nevertheless, it is not necessary that the series exist in order that correntropy exist.…”

“…If its value is too large, correntropy will basically rely on second-order properties. On the other hand, if the value is too small, an undesirable behavior can be observed, in which the correntropy is dominated by moments of extremely highorder [4]. In that sense, σ plays a different role in correntropy, being related to weights on the statistical moments, while, in the information potential, σ is closely related to the shape of the distributions.…”

New methods for adaptive equalization based on Information Theoretic Learning

“…The criteria were presented encompassing aspects like the analytical calculus of the costs and their estimation, as well as their gradient calculus for the application in gradientbased methods for optimization. It is important to emphasize that the described method for analytical computation of the correntropy is also a contribution of this work (in the literature, the correntropy is analytically analyzed in terms of its Taylor series expansion [Principe, 2010;Yang et al, 2011]). Next, some statistical estimation issues like number of samples, number of delays and noise disturbances were analyzed, encompassing all considered criteria.…”