Learning function from structure in neuromorphic networks

Abstract

The connection patterns of neural circuits in the brain form a complex network. Collective signaling within the network manifests as patterned neural activity, and is thought to support human cognition and adaptive behavior. Recent technological advances permit macro-scale reconstructions of biological brain networks. These maps, termed connectomes, display multiple non-random architectural features, including heavy-tailed degree distributions, segregated communities and a densely interconnected core. Yet, how computation and functional specialization emerge from network architecture remains unknown. Here we reconstruct human brain connectomes using in vivo diffusion-weighted imaging, and use reservoir computing to implement these connectomes as artificial neural networks. We then train these neuromorphic networks to learn a cognitive task. We show that biologically realistic neural architectures perform optimally when they display critical dynamics. We find that performance is driven by network topology, and that the modular organization of large-scale functional systems is computationally relevant. Throughout, we observe a prominent interaction between network structure and dynamics, such that the same underlying architecture can support a wide range of learning capacities across dynamical regimes. This work opens new opportunities to discover how the network organization of the brain optimizes cognitive capacity, conceptually bridging neuroscience and artificial intelligence.

Brain Connectivity meets Reservoir Computing

“…This points out once again to the importance of random wiring diagrams for ESNs' performance. This fact is as well in line with a recent study using human connectivity as reservoir of ESNs, which showed that random connectivity indeed achieved globally maximal performances across almost all tested hyperparameters, provided the wiring cost is not considered [24]. The functional importance of randomness is also consistent with the fact that stochastic processes play a fundamental role in brain connectivity formation, both at a micro and meso/macro-scale, as supported by empirical [25], and computational modeling studies [26,27].…”

“…This points out once again to the importance of random wiring diagrams for ESNs’ performance. This fact is as well in line with a recent study using human connectivity as reservoir of ESNs, which showed that random connectivity indeed achieved globally maximal performances across almost all tested hyperparameters, provided the wiring cost is not considered [24].…”

“…As we aimed at testing the potential impact of the global wiring diagram of connectomes, we consider the entire connectomes as one unique network to create the reservoirs. This is different from a previous study where the connectivity was divided into subnetworks corresponding to brain systems that were separately trained [24]. We decided to avoid here the strong assumptions that such an approach implies, but we recognize a potential for future studies in the direction, for example, exploring the division of networks as different input/output subsystems.…”

Constructing neural network models from brain data reveals representational transformations underlying adaptive behavior

“…Building on this work, we recently proposed that the functional network architecture of the brain can be used to build network coding models -models of brain function that describe the encoding and decoding of task-relevant brain activity constrained by connectivity 49 . Related proposals have also been put forward in the electron microscopy connectomics literature, suggesting that structural wiring diagrams of the brain can inform functional models of biological systems (e.g., the drosophila's visual system or the human brain's intrinsic memory capacity) 47,50,51 . In addition, work in mean-field network models have revealed a direct link between connectivity and computations, finding that low-dimensional connectivity patterns (which also exist in fMRI data 52 ) are useful for performing tasks 53 .…”

Bio-instantiated recurrent neural networks

“…In sum, the network topology pertaining to a plethora of BNNs are in contrast to the handcrafted engineering-driven architecture of ANNs. A recent study examines the effect of constructing RNNs with the empirically discerned topology of the human brain network (Suarez et al, 2020). This study, however, examines a different class of RNNs (echo state networks) than we examine in our approach (Elman networks), and in addition, echo state networks are trained with a different algorithm as the one that is used here (backpropagation-through-time).…”