A Unifying Framework of Synaptic and Intrinsic Plasticity in Neural Populations

Abstract

A neuronal population is a computational unit that receives a multivariate, time-varying input signal and creates a related multivariate output. These neural signals are modeled as stochastic processes that transmit information in real time, subject to stochastic noise. In a stationary environment, where the input signals can be characterized by constant statistical properties, the systematic relationship between its input and output processes determines the computation carried out by a population. When these statistical characteristics unexpectedly change, the population needs to adapt to its new environment if it is to maintain stable operation. Based on the general concept of homeostatic plasticity, we propose a simple compositional model of adaptive networks that achieve invariance with regard to undesired changes in the statistical properties of their input signals and maintain outputs with well-defined joint statistics. To achieve such invariance, the network model combines two functionally distinct types of plasticity. An abstract stochastic process neuron model implements a generalized form of intrinsic plasticity that adapts marginal statistics, relying only on mechanisms locally confined within each neuron and operating continuously in time, while a simple form of Hebbian synaptic plasticity operates on synaptic connections, thus shaping the interrelation between neurons as captured by a copula function. The combined effect of both mechanisms allows a neuron population to discover invariant representations of its inputs that remain stable under a wide range of transformations (e.g., shifting, scaling and (affine linear) mixing). The probabilistic model of homeostatic adaptation on a population level as presented here allows us to isolate and study the individual and the interaction dynamics of both mechanisms of plasticity and could guide the future search for computationally beneficial types of adaptation.

In vitro neural networks minimise variational free energy

“…These form observations speak to the biological plausibility of the variational message passing scheme used to simulate neural responses in this paper. Although several biologically plausible blind source separation methods in the continuous state space have been developed 44 – 48 , to our knowledge, this is the first attempt to explain neuronal blind source separation using a biologically plausible learning algorithm in the discrete (binary) state space.…”

A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems

“…This intrinsic probabilistic property of memristive devices can be exploited for implementing stochastic learning in neuromorphic architectures 43,44,48,[64][65][66] , which in turn can be used to implement faithful models of biological cortical microcircuits 67,68 , solve memory capacity and classification problems in artificial neural network applications 69,70 , and reduce the network sensitivity to their variability 43 . Recent results on stochastic learning modulated by regularization mechanisms, such as homeostasis or intrinsic plasticity 44,[71][72][73] , present an excellent potential for exploiting the features memristive devices, even when restricted to binary values. e. Don't (hard) limit your devices.…”

On the Achievability of Blind Source Separation for High-Dimensional Nonlinear Source Mixtures

“…Nonetheless, the manner in which the brain can possibly solve a nonlinear BSS problem remains unclear, even though it might be a prerequisite for many of its cognitive processes such as visual recognition (DiCarlo et al, 2012). While Oja's subspace rule for PCA (Oja, 1989) and Amari's ICA algorithm (Amari et al, 1996) were used in this article, these rules can be replaced with more biologically plausible local Hebbian learning rules (Foldiak, 1990;Linsker, 1997;Pehlevan, Mohan, & Chklovskii, 2017;Isomura & Toyoizumi, 2016Leugering & Pipa, 2018) that require only directly accessible signals to update synapses. A recent work indicated that even a single-layer neural network can perform both PCA and ICA through a local learning rule (Isomura & Toyoizumi, 2018), implying that even a single-layer network can perform a nonlinear BSS.…”

“…Our theorems indicate that a combination of PCA and ICA algorithms can reliably separate hidden sources that are nonlinearly mixed in the environment, when sufficiently rich sensory inputs are provided. While we have used Oja's subspace rule for PCA and Amari's ICA rule in this paper, more biologically plausible local Hebbian learning rules are proposed [59][60][61][62][63][64]. A recent work showed that even a single-layer neural network can perform both PCA and ICA through a local learning rule [63].…”