A quick and easy way to estimate entropy and mutual information for neuroscience

Abstract

Calculations of entropy of a signal or mutual information between two variables are valuable analytical tools in the field of neuroscience. They can be applied to all types of data, capture nonlinear interactions and are model independent. Yet the limited size and number of recordings one can collect in a series of experiments makes their calculation highly prone to sampling bias. Mathematical methods to overcome this so called “sampling disaster” exist, but require significant expertise, great time and computational costs. As such, there is a need for a simple, unbiased and computationally efficient tool for reliable entropy and mutual information estimation. In this paper, we propose that application of entropy-coding compression algorithms widely used in text and image compression fulfill these requirements. By simply saving the signal in PNG picture format and measuring the size of the file on the hard drive, we can reliably estimate entropy through different conditions. Furthermore, with some simple modifications of the PNG file, we can also estimate mutual information between a stimulus and the observed responses into multiple trials. We show this using White noise signals, electrophysiological signals and histological data. Although this method does not give an absolute value of entropy or mutual information, it is mathematically correct, and its simplicity and broad use make it a powerful tool for their estimation through experiments.

A Quick and Easy Way to Estimate Entropy and Mutual Information for Neuroscience

“…We thank Q. Bernard, C. Leterrier, V. Marra, T. Jensen, K. Zheng, S. Martiniani, and R. Beck for helpful discussions. A first version of this manuscript has been released on bioRxiv ( Zbili and Rama, 2020 ), https://www.biorxiv.org/content/10.1101/2020.08.04.236174v2 .…”

Estimating the entropy of neural data by saving them as a .png file

“…Third, he study extends the use of the method to imaging data of neuronal morphology, such as charting the growth stage of neuronal cultures. However, the radial entropy of a dendritic tree seems at first more difficult to interpret than the common Sholl analysis of radial crossings of dendrite segments (Figure 6Ac of Zbili and Rama, 2021). As the authors note, a similar technique is used in paleobiology to discriminate pictures of biogenic rocks from abiogenic ones (Wagstaff and Corsetti, 2010).…”