Sparse low-order interaction network underlies a highly correlated and learnable neural population code

Abstract

Information is carried in the brain by the joint activity patterns of large groups of neurons. Understanding the structure and function of population neural codes is challenging because of the exponential number of possible activity patterns and dependencies among neurons. We report here that for groups of ∼100 retinal neurons responding to natural stimuli, pairwise-based models, which were highly accurate for small networks, are no longer sufficient. We show that because of the sparse nature of the neural code, the higher-order interactions can be easily learned using a novel model and that a very sparse low-order interaction network underlies the code of large populations of neurons. Additionally, we show that the interaction network is organized in a hierarchical and modular manner, which hints at scalability. Our results suggest that learnability may be a key feature of the neural code.high-order | correlations | maximum entropy | neural networks | sparseness S ensory and motor information is carried in the brain by sequences of action potentials of large populations of neurons (1-3) and, often, by correlated patterns of activity (4-11). The detailed nature of the code of neural populations, namely the way information is represented by the specific patterns of spiking and silence over a group of neurons, is determined by the dependencies among cells. For small groups of neurons, we can directly sample the full distribution of activity patterns of the population; identify all the underlying interactions, or lack thereof; and understand the design of the code (12-15). However, this approach cannot work for large networks: The number of possible activity patterns of just 100 neurons, a population size that already has clear functional implications (16), exceeds 10 30 . Thus, our understanding of the code of large neural populations depends on finding simple sets of dependencies among cells that would capture the network behavior (17)(18)(19).The success of pairwise-based models in describing the strongly correlated activity of small groups of neurons (19-25) suggests one such simplifying principle of network organization and population neural codes, which also simplifies their analysis. Using only a quadratic number of interactions, out of the exponential number of potential ones, pairwise maximum entropy models reveal that the code relies on strongly correlated network states and exhibits distributed error-correcting structure (19,21). It is unclear, however, if pairwise models are sufficient for large networks, particularly when presented with natural stimuli that contain high-order correlation structure. Here we show that in this case pairwise models capture much, but not all, of the network behavior. This implies a much more complicated structure of population codes (26,27). Because learning even pairwise models is computationally hard (28-33), this may seem to suggest that population codes would be extremely hard to learn.We show here that this is not the case for neural population codes. The sparseness of...

Higher-Order Interactions Characterized in Cortical Activity

“…This is consistent with recent findings that the inclusion of higher-order interactions yielded better approximation of LFP patterns (Santos et al, 2010) as well as neuronal synchrony (Montani et al, 2009). Our findings are also in line with the shortcomings reported for the Ising model to predict correlated activity of nearby neurons (Ohiorhenuan et al, 2010;Ohiorhenuan and Victor, 2011) or of large network activity (ϳ100 retina ganglion cells) in response to natural stimuli (Ganmor et al, 2011b). Moreover, our analysis provides new insights into these recent findings.…”

“…9) provides a possible explanation for the phenomenon that the more pronounced third-order interactions are usually accompanied by strong pairwise interaction or correlation (Ohiorhenuan and Victor, 2011). In addition, the same mechanism may account for the specific failure of the Ising model to capture responses to natural stimuli (Ganmor et al, 2011b), as such responses are often characterized by sparse firing and strong pairwise correlations (Stüttgen and Schwarz, 2008;Jadhav et al, 2009). More generally, our results give a satisfactory account for the reported lack of higher-order interactions in many previous studies (Schneidman et al, 2006;Shlens et al, 2006;Tang et al, 2008;Yu et al, 2008), as those interactions are most pronounced in the regime of low rates and high pairwise correlation, which these studies often did not discriminate.…”

“…Schneidman et al (2006) and Shlens et al (2006) showed that pairwise interactions explain most of the spiking activity in small (Յ10 neurons) retina networks, a finding that was soon extended to include larger systems (Cocco et al, 2009;Shlens et al, 2009;Ganmor et al, 2011a), in vivo cortical networks (Yu et al, 2008;Ohiorhenuan et al, 2010), and correlations among population activities measured in the local field potentials (LFPs) (Tang et al, 2008;Santos et al, 2010). In contrast, a number of more recent studies (Montani et al, 2009;Ohiorhenuan et al, 2010;Santos et al, 2010;Ohiorhenuan and Victor, 2011;Ganmor et al, 2011b) have identified significant higher-order interactions in neuronal spiking and LFP activities, e.g., third-order interactions among closely neighbored (Ͻ300 m) cortical neurons (Ohiorhenuan et al, 2010;Ohiorhenuan and Victor, 2011) and interactions of up to the eighth order among retinal ganglion cells (Ganmor et al, 2011b). Clearly, the structure of higher-order interactions and their contributions to cortical dynamics are still open to debate.…”

Thermodynamics and signatures of criticality in a network of neurons

“…The maximum entropy approach to networks of neurons has been explored, in several different systems, for nearly a decade (14)(15)(16)(17)(18)(19)(20)(21)(22)(23), and there have been parallel efforts to use this approach in other biological contexts (24)(25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35). Recently, we have used the maximum entropy method to build models for the activity of up to N = 120 neurons in the experiments described above (11); see Fig.…”

Reverse-engineering biological networks from large data sets

“…These interactions are typically very specific, and highly coordinated spatially and temporally [4][5][6][7][8]. Involving not just pairs, but also larger groups of components acting in concert [9][10][11][12][13][14], they are responsible for the rich diversity of complex phenomena and behaviors that make living systems work. Although often prohibitively numerous to model individually (though see [15]), these components and their corresponding interactions can be represented formally as graphs [16], known colloquially as biological networks [3,[17][18][19][20][21][22][23].…”