Applying Information Theory to Neuronal Networks: From Theory to Experiments

Abstract

Information-theory is being increasingly used to analyze complex, self-organizing processes on networks, predominantly in analytical and numerical studies. Perhaps one of the most paradigmatic complex systems is a network of neurons, in which cognition arises from the information storage, transfer, and processing among individual neurons. In this article we review experimental techniques suitable for validating information-theoretical predictions in simple neural networks, as well as generating new hypotheses. Specifically, we focus on techniques that may be used to measure both network (microcircuit) anatomy as well as neuronal activity simultaneously. This is needed to study the role of the network structure on the emergent collective dynamics, which is one of the reasons to study the characteristics of information processing. We discuss in detail two suitable techniques, namely calcium imaging and the application of multi-electrode arrays to simple neural networks in culture, and discuss their advantages and limitations in an accessible manner for non-experts. In particular, we show that each technique induces a qualitatively different type of error on the measured mutual information. The ultimate goal of this work is to bridge the gap between theorists and experimentalists in their shared goal of understanding the behavior of networks of neurons. OPEN ACCESSEntropy 2014, 16 5722

Upper Limit on the Thermodynamic Information Content of an Action Potential

“…At a maximum, H = 1 bit. Numerous hypotheses, models, and theories use the above equation for the Shannon entropy of a binary random variable as a starting point for more sophisticated analyses of neuronal information content (Borst and Theunissen, 1999;Arcas et al, 2003;Victor, 2006;Sharpee and Bialek, 2007;Jensen et al, 2013;Jung et al, 2014;Sengupta et al, 2014). For example, the popular "direct method" for calculating the information content of a spike train begins by dividing its total duration into a number of evenly spaced time bins.…”

The dependence of neuronal encoding efficiency on Hebbian plasticity and homeostatic regulation of neurotransmitter release

“…Moreover, the diversity of neural spiking strongly depends on the properties of their synapses which remarkably vary in different types of neurons. This quantity is usually calculated using “mutual information” between a spike train and the stimulus as an information theoretic approach (Kumbhani et al, 2007 ; Faghihi et al, 2013 ; Fan, 2014 ; Jung et al, 2014 ). As neural systems should be able to detect a fluctuation in a stimulus intensity, a new encoding efficiency measure has been recently introduced which uses the geometric distance between stimulus and the response of a given neuron (Faghihi and Moustafa, 2015 ).…”