Mean First Passage Memory Lifetimes by Reducing Complex Synapses to Simple Synapses

2017

Neural Computation

Self Cite

Memory models based on synapses with discrete and bounded strengths store new memories by forgetting old ones. Memory lifetimes in such memory systems may be defined in a variety of ways. A mean first passage time (MFPT) definition overcomes much of the arbitrariness and many of the problems associated with the more usual signal-to-noise ratio (SNR) definition. We have previously computed MFPT lifetimes for simple, binary-strength synapses that lack internal, plasticity-related states. In simulation we have also seen that for multistate synapses, optimality conditions based on SNR lifetimes are absent with MFPT lifetimes, suggesting that such conditions may be artifactual. Here we extend our earlier work by computing the entire first passage time (FPT) distribution for simple, multistate synapses, from which all statistics including the MFPT lifetime may be extracted. For this, we develop a Fokker-Planck equation using the jump moments for perceptron activation. Two models are considered that satisfy a particular eigenvector condition that this approach requires. In these models, MFPT lifetimes do not exhibit optimality conditions, while in one but not the other, SNR lifetimes do exhibit optimality. Thus, not only are such optimality conditions artifacts of the SNR approach, but they are also strongly model-dependent. By examining the variance in the FPT distribution, we may identify regions in which memory storage is subject to high variability, although MFPT lifetimes are nevertheless robustly positive.In such regions, SNR lifetimes are typically (defined to be) zero. FPT-defined memory lifetimes therefore provide an analytically superior approach and also have the virtue of being directly related to a neuron's firing properties.2

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

First Passage Time Memory Lifetimes for Simple, Multistate Synapses

2017

Neural Computation

Self Cite

“…FPT approaches to memory lifetimes with complex synapses are analytically hard. For the specific case of binary-strength, ν = 2 synapses, we examined MFPTs by developing methods that essentially integrate out the internal states, thereby replacing full transition matrices by simpler, 2 × 2 matrices (Elliott, 2017a). The price paid for this reduction of complex synapses to simple synapses is that the transition matrices become time-or memory-storage-step-dependent.…”

Section: Discussionmentioning

confidence: 99%

“…We have introduced a first-passage-time (FPT) approach to defining memory lifetimes in a feedforward setting with a single perceptron for simplicity (Elliott, 2014(Elliott, , 2017a. Specifically, we define a memory's lifetime as the mean first passage time (MFPT) for the perceptron's activation to fall below firing threshold, and we may also consider the variability in this lifetime by examining the higher-order statistics of the FPT distribution.…”

Section: Introductionmentioning

confidence: 99%

“…It prevented us from analysing two models discussed in detail below that do not satisfy this condition. Moreover, it prevents us from extending the approach from simple to complex synapses with internal states, where integrating out the internal states leads to time-dependent transition matrices whose eigenvectors will therefore also be time-dependent (Elliott, 2017a). Here, therefore, we develop an alternative approach to the analysis of FPT memory lifetimes that dispenses with the eigenvector requirement.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

First Passage Time Memory Lifetimes for Simple, Multistate Synapses: Beyond the Eigenvector Requirement

2019

Neural Computation

Self Cite

Models of associative memory with discrete-strength synapses are palimpsests, learning new memories by forgetting old ones. Memory lifetimes can be defined by the mean first passage time (MFPT) for a perceptron's activation to fall below firing threshold. By imposing the condition that the vector of possible strengths available to a synapse is a left eigenvector of the stochastic matrix governing transitions in strength, we previously derived results for MFPTs and first passage time (FPT) distributions in models with simple, multistate synapses. This condition permits jump moments to be computed via a 1-dimensional Fokker-Planck approach. Here, we study memory lifetimes in the absence of this condition. To do so, we must introduce additional variables, including the perceptron activation, that parameterize synaptic configurations, permitting Markovian dynamics in these variables to be formulated. FPT problems in these variables require solving multidimensional partial differential or integral equations. However, the FPT dynamics can be analytically well approximated by focusing on the slowest eigenmode in this higher-dimensional space. We may also obtain a much better approximation by restricting to the two dominant variables in this space, the restriction making numerical methods tractable. Analytical and numerical methods are in excellent agreement with simulation data, validating our methods. These methods prepare the ground for the study of FPT memory lifetimes with complex rather than simple, multistate synapses.

The Impact of Sparse Coding on Memory Lifetimes in Simple and Complex Models of Synaptic Plasticity

2022

Biol Cybern

Self Cite

Models of associative memory with discrete state synapses learn new memories by forgetting old ones. In the simplest models, memories are forgotten exponentially quickly. Sparse population coding ameliorates this problem, as do complex models of synaptic plasticity that posit internal synaptic states, giving rise to synaptic metaplasticity. We examine memory lifetimes in both simple and complex models of synaptic plasticity with sparse coding. We consider our own integrative, filter-based model of synaptic plasticity, and examine the cascade and serial synapse models for comparison. We explore memory lifetimes at both the single-neuron and the population level, allowing for spontaneous activity. Memory lifetimes are defined using either a signal-to-noise ratio (SNR) approach or a first passage time (FPT) method, although we use the latter only for simple models at the single-neuron level. All studied models exhibit a decrease in the optimal single-neuron SNR memory lifetime, optimised with respect to sparseness, as the probability of synaptic updates decreases or, equivalently, as synaptic complexity increases. This holds regardless of spontaneous activity levels. In contrast, at the population level, even a low but nonzero level of spontaneous activity is critical in facilitating an increase in optimal SNR memory lifetimes with increasing synaptic complexity, but only in filter and serial models. However, SNR memory lifetimes are valid only in an asymptotic regime in which a mean field approximation is valid. By considering FPT memory lifetimes, we find that this asymptotic regime is not satisfied for very sparse coding, violating the conditions for the optimisation of single-perceptron SNR memory lifetimes with respect to sparseness. Similar violations are also expected for complex models of synaptic plasticity.