Dual stochasticity in the cortex as a biologically plausible learning with the most efficient coding

Abstract

Neurons and synapses in the cerebral cortex behave stochastically even during precise perception and reliable learning of animals. While studies have revealed advantages of these stochastic features, relationship and synergy of them remain elusive. Here, we show that these stochastic features are unified into a framework to provide an efficient and biologically-plausible learning algorithm that consistently explains various experimental findings of the brain, which includes statistics of cortical circuit and the most efficient power-low encoding of cortex. The algorithm can be regarded as an integration of major branches of existing algorithms of neural networks, back propagation learning and Boltzmann machine, while it overcomes known limitations of them. As an experimentally testable prediction, slow retrograde modulation of excitability of neurons is also derived. We anticipate that the algorithm is, due to its simplicity and flexibility, beneficial to develop neuromorphic devices or scalable AI-chips, and bridges the gap between neuroscience and machine learning.

Computational Roles of Intrinsic Synaptic Dynamics

“…Another interesting perspective was introduced by integrating stochasticity at the synaptic and neuronal levels to draw samples from the encoded distribution [75]. Here, it was shown that the fluctuations of both neural activity and synaptic weights work as the Gibbs sampling [76], generalizing learning in Boltzmann machines [77] with additional weight sampling.…”

“…Another recent work by Teramae [71] integrated two different timescales of sampling in synapse and neuron. It generalized Boltzmann machine [72] learning with additional weight sampling, and proposed that fluctuations of both neural activity and synaptic weights arose as a result of the Gibbs sampling [73].…”