Application of offset estimator of differential entropy and mutual information with multivariate data

Abstract

Numerical estimators of differential entropy and mutual information can be slow to converge as sample size increases. The offset Kozachenko–Leonenko (KLo) method described here implements an offset version of the Kozachenko–Leonenko estimator that can markedly improve convergence. Its use is illustrated in applications to the comparison of trivariate data from successive scene color images and the comparison of univariate data from stereophonic music tracks. Publicly available code for KLo estimation of both differential entropy and mutual information is provided for R, Python, and MATLAB computing environments at https://github.com/imarinfr/klo.

Little information loss with red-green color deficient vision in natural environments

“…(1989) 104 https://sites.uef.fi/spectral/munsell-colors-matt-spectrofotometer-measured/ Human cone pigment spectral sensitivities Stockman and Sharpe (2000) 105 http://cvrl.ioo.ucl.ac.uk/ Cone pigment shift routine Foster, D.H. (2010) 106 https://doi.org/10.5281/zenodo.8121909 Offset version of the Kozachenko-Leonenko estimator of mutual information Marín-Franch et al. (2022) 107 https://github.com/imarinfr/klo …”

“…Mutual information was estimated numerically with an offset version 111 , 107 of the Kozachenko-Leonenko k th-nearest-neighbor estimator, 127 , 128 which converges relatively rapidly and accurately with increasing sample size. 111 , 107 The number of sample points taken from each image in the set of 50 and set of 100 images was 10 4 and the maximum available were taken from the color palettes, namely ∼10 3 . Resampling was used to test the stability of the estimates of .…”