Efficient and robust coding in heterogeneous recurrent networks

Abstract

Cortical networks show a large heterogeneity of neuronal properties. However, traditional coding models have focused on homogeneous populations of excitatory and inhibitory neurons. Here, we analytically derive a class of recurrent networks of spiking neurons that close to optimally track a continuously varying input online, based on two assumptions: 1) every spike is decoded linearly and 2) the network aims to reduce the meansquared error between the input and the estimate. From this we derive a class of predictive coding networks, that unifies encoding and decoding and in which we can investigate the difference between homogeneous networks and heterogeneous networks, in which each neurons represents different features and has different spike-generating properties. We find that in this framework, 'type 1' and 'type 2' neurons arise naturally and networks consisting of a heterogeneous population of different neuron types are both more efficient and more robust against correlated noise. We make two experimental predictions: 1) we predict that integrators show strong correlations with other integrators and resonators are correlated with resonators, whereas the correlations are much weaker between neurons with different coding properties and 2) that 'type 2' neurons are more coherent with the overall network activity than 'type 1' neurons. Efficient coding | Filter networks | Recurrent networks | Type 1 and type 2 neurons | Trial-to-trial variability | Robustness against noise | Signal and noise correlationsCorrespondence: f.zeldenrust@neurophysiology.nl

Neural heterogeneity promotes robust learning

“…The brain is known to be deeply heterogeneous at all scales (Koch and Laurent 1999), but it is still not known whether this heterogeneity plays an important functional role or if it is just a byproduct of noisy developmental processes and contingent evolutionary history. A number of hypothetical roles have been suggested (reviewed in (Gjorgjieva, Drion, and Marder 2016)), in efficient coding (Shamir and Sompolinsky 2006; Chelaru and Dragoi 2008; Osborne et al 2008; Marsat and Maler 2010; Padmanabhan and Urban 2010; Hunsberger, Scott, and Eliasmith 2014; Zeldenrust, Gutkin, and Denéve 2019), reliability (Lengler, Jug, and Steger 2013), working memory (Kilpatrick, Ermentrout, and Doiron 2013), and functional specialisation (Duarte and Morrison 2019). However, previous studies have largely used simplified tasks or networks, and it remains unknown whether or not heterogeneity can help animals solve complex information processing tasks in natural environments.…”

“…The brain is known to be deeply heterogeneous at all scales [1], but it is still not known whether this heterogeneity plays an important functional role or if it is just a byproduct of noisy developmental processes and contingent evolutionary history. A number of hypothetical roles have been suggested (reviewed in [2]), in efficient coding [3–9], reliability [10], working memory [11], and functional specialisation [12]. However, previous studies have largely used simplified tasks or networks, and it remains unknown whether or not heterogeneity can help animals solve complex information processing tasks in natural environments.…”

The tuning of tuning: how adaptation influences single cell information transfer

“…The functional reason for this, remains an open topic. One theory suggests that such fast inhibition is needed for predictive coding (Boerlin et al, 2013; Denève and Machens, 2016; Hawkins and Ahmad, 2017; Zeldenrust et al, 2019). In addition to this, we suggest that the observed compression of excitatory neurons and the broadband information transfer of inhibitory neurons serve a common goal: sparse coding (Földiák, 1990; Foldiak and Endres, 2008; Olshausen and Field, 1996).…”

Compensatory variability in network parameters enhances memory performance in theDrosophilamushroom body

“…Indeed, previous work has emphasized the benefits of neuronal heterogeneity rather than neuronal homogeneity [12–14]. Of course, different neuronal classes encode different information (e.g., visual vs. auditory neurons, or ON vs. OFF cells).…”

“…However, less attention has been paid to potential benefits of maintaining consistent neuronal properties across a population of neurons within an individual circuit. Indeed, previous work has emphasized the benefits of neuronal heterogeneity rather than neuronal homogeneity [12][13][14]. Of course, different neuronal classes encode different information (e.g., visual vs. auditory neurons, or ON vs. OFF cells).…”