Optimal Information Storage and the Distribution of Synaptic Weights

Abstract

It is widely believed that synaptic modifications underlie learning and memory. However, few studies have examined what can be deduced about the learning process from the distribution of synaptic weights. We analyze the perceptron, a prototypical feedforward neural network, and obtain the optimal synaptic weight distribution for a perceptron with excitatory synapses. It contains more than 50% silent synapses, and this fraction increases with storage reliability: silent synapses are therefore a necessary byproduct of optimizing learning and reliability. Exploiting the classical analogy between the perceptron and the cerebellar Purkinje cell, we fitted the optimal weight distribution to that measured for granule cell-Purkinje cell synapses. The two distributions agreed well, suggesting that the Purkinje cell can learn up to 5 kilobytes of information, in the form of 40,000 input-output associations.

Cerebellar LTD and Pattern Recognition by Purkinje Cells

“…In the simulations that were described previously (Figures 1, 2, and 6), we presented PF activity patterns in which 1000 of the 147,400 synapses were activated synchronously. This synchronous activity level of approximately 0.7% is similar to previous estimates of approximately 1% (Albus, 1971; Marr, 1969; Schweighofer and Ferriol, 2000) and is in agreement with a recent study showing optimal performance for activity levels of 0.2%–1% (Brunel et al., 2004). We studied the robustness of our predictions by varying the PF activity level and found that the predictions of the model held for patterns with at least 750 and not more than 8000 synchronously active synapses (Figure S1).…”

“…A recent study of pattern storage in Purkinje cells (Brunel et al., 2004) has suggested that each Purkinje cell could learn to recognize approximately 40000 different PF input patterns. However, this is on the basis of the assumptions that the Purkinje-cell output is binary and that the combined noise sources result in a SD of σ ≤ 0.4 mV.…”

Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits

“…For example, in the feed-forward projection from granule to Purkinje cells in the cerebellum, the distribution was fitted by a truncated Gaussian distribution, argued to be optimal for information storage [40]. It would be interesting to see if analogous theory could be developed to explain the lognormal distribution seen among the layer 5 pyramid recurrent connections.…”

Inhibition gates supralinear Ca2+ signaling in Purkinje cell dendrites during practiced movements

“…This supralinearity may reflect a change in dendritic excitability that facilitates the propagation of climbing fiber-evoked Ca 2+ spikes into spiny branchlets yielding additional Ca 2+ entry (Otsu et al, 2014; but see Wang et al, 2000). Parallel fibers also activate MLIs, driving rapid feed-forward inhibition that attenuates parallel fiber excitation of PCs (Brunel et al, 2004; Mittmann et al, 2005). We reasoned that, by reducing dendritic excitability, feed-forward inhibition could diminish the ability of parallel fibers to enhance subsequent climbing fiber-evoked Ca 2+ responses and thus provide a mechanism to explain in vivo gating of non-linear dendritic Ca 2+ signaling in PCs during licking movements.…”

“…However, if self-generated parallel fiber activity is sufficient to enhance ongoing climbing fiber Ca 2+ signals, then PCs would continuously undergo plasticity despite conditions where learning provides no benefit to motor outcomes. To counteract direct parallel fiber excitation of PCs, parallel fibers also excite molecular layer interneurons (MLIs) driving feed-forward inhibition that can attenuate parallel fiber excitatory postsynaptic potentials (EPSPs) (Brunel et al, 2004; Mittmann et al, 2005). MLIs can also directly reduce climbing fiber-evoked responses (Callaway et al, 1995; Kitamura and Häusser, 2011) and impair LTD at parallel fiber-PC synapses (Ekerot and Kano, 1985).…”