Learning probabilistic neural representations with randomly connected circuits

Abstract

The brain represents and reasons probabilistically about complex stimuli and motor actions using a noisy, spike-based neural code. A key building block for such neural computations, as well as the basis for supervised and unsupervised learning, is the ability to estimate the surprise or likelihood of incoming high-dimensional neural activity patterns. Despite progress in statistical modeling of neural responses and deep learning, current approaches either do not scale to large neural populations or cannot be implemented using biologically realistic mechanisms. Inspired by the sparse and random connectivity of real neuronal circuits, we present a model for neural codes that accurately estimates the likelihood of individual spiking patterns and has a straightforward, scalable, efficient, learnable, and realistic neural implementation. This model’s performance on simultaneously recorded spiking activity of >100 neurons in the monkey visual and prefrontal cortices is comparable with or better than that of state-of-the-art models. Importantly, the model can be learned using a small number of samples and using a local learning rule that utilizes noise intrinsic to neural circuits. Slower, structural changes in random connectivity, consistent with rewiring and pruning processes, further improve the efficiency and sparseness of the resulting neural representations. Our results merge insights from neuroanatomy, machine learning, and theoretical neuroscience to suggest random sparse connectivity as a key design principle for neuronal computation.

Sequential and efficient neural-population coding of complex task information

“…Our point here about the theoretical utility of random sequential neural codes extends the literature on how neural circuits can exploit random designs to perform interesting computational functions, such as separable neural representations (Fusi, Miller, and Rigotti 2016; Rigotti et al 2013; Babadi and Sompolinsky 2014; Lindsay et al 2017), short term memory of input patterns and dynamics (S. Ganguli and Sompolinsky 2010; Charles, Yin, and Rozell 2017; Jaeger and Haas 2004; Bouchacourt and Buschman 2019), and unsupervised learning of the structure of input signals (Maoz et al 2020).…”

A Neuroscience-Inspired Spiking Neural Network for Auditory Spatial Attention Detection Using Single-Trial EEG

“…One example of this class of SNNs is a model for neural coding that is inspired by sparse coding and random connectivity of by real neural circuits where structural changes in the random connectivity induced by pruning process. In this work [44] random sparse connectivity has been presented as a key principle of cortical computation. In order to implement pruning, a STDP (Spike-Time Dependent Plasticity) model has been proposed [45].…”

Predicting perturbation effects from resting activity using functional causal flow

“…Alternatives such as inverse methods based on Ising models utilize time-consuming learning schemes (Tkacik et al, 2006) though recently faster algorithms have been proposed (Cocco et al, 2009, Maoz et al, 2020). Other approaches applicable to spike trains include generalized linear models (Pillow et al, 2008) or spike train cross-correlograms (English et al, 2017).…”

“…Existing methods for estimating functional interactions between multi-dimensional time series include linear regression (79), Granger causality (GC) (80), and inter-areal coherence (81,82). (16,88). Other approaches applicable to spike trains include generalized linear models (89) or spike train cross-correlograms (90).…”