Analytic solution of attractor neural networks on scale-free graphs

Castillo, Isaac Pérez; Wemmenhove, B.; Hatchett, J. P. L.; Coolen, A. C. C.; Skantzos, N. S.; Nikoletopoulos, T

doi:10.1088/0305-4470/37/37/002

Cited by 35 publications

(46 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Davey et al also give results for small-world networks of perceptrons (Davey et al, 2006). Other investigations have been reported in which sparse Hopfield networks are connected with scale-free connection graphs, for example (da Fontoura Costa, and Stauffer, 2003;Kosinski, and Sinolecka, 1999;Perez Castillo, Wemmenhove, Hatchett, Coolen, Skantzos, and Nikoletopoulos, 2004;Stauffer, Aharony, da Fontoura Costa, and Adler, 2003;Torres, Munoz, Marro, and Garrido, 2004). Another approach is to build networks with a modular structure: see for example (Horn, Levy, and Ruppin, 1999;Renart, Parga, and Rolls, 1999).…”

Section: Related Workmentioning

confidence: 99%

Efficient architectures for sparsely-connected high capacity associative memory models

2007

View full text Add to dashboard Cite

In physical implementations of associative memory, wiring costs play a significant role in shaping patterns of connectivity. In this study of sparsely-connected associative memory, a range of architectures is explored in search of optimal connection strategies which maximise pattern-completion performance, while at the same time minimising wiring costs. It is found that architectures in which the probability of connection between any two nodes is based on relatively tight Gaussian and exponential distributions perform well, and that for optimum performance, the width of the Gaussian distribution should be made proportional to the number of connections per node. It is also established from a study of other connection strategies that distal connections are not necessary for good pattern-completion performance. Convergence times and network scalability are also addressed in the wide ranging study.

show abstract

Section: Related Workmentioning

confidence: 99%

Efficient architectures for sparsely-connected high capacity associative memory models

2007

View full text Add to dashboard Cite

show abstract

“…In particular due to the unexpectedly rich and varied range of multi-disciplinary applications of finite connectivity replica techniques which emerged subsequently in, for example, spin-glass modelling [6][7][8][9], error correcting codes [10][11][12][13], theoretical computer science [14][15][16][17], recurrent neural networks [18][19][20] and 'small-world' networks [21], this field is presently enjoying a renewed interest and popularity. Until very recently, analysis was limited to the equilibrium properties of such models, but now attention has also turned to the dynamics of finitely connected spin systems [22][23][24][25], using combinatorial and generating functional methods.…”

Section: Introductionmentioning

confidence: 99%

Finitely connected vector spin systems with random matrix interactions

Coolen¹,

Skantzos²,

Castillo³

et al. 2005

J. Phys. A: Math. Gen.

Self Cite

View full text Add to dashboard Cite

We use finite connectivity equilibrium replica theory to solve models of finitely connected unit-length vectorial spins, with random pair-interactions which are of the orthogonal matrix type. Finitely connected spin models, although still of a mean-field nature, can be regarded as a convenient level of description in between fully connected and finite-dimensional ones. Since the spins are continuous and the connectivity c remains finite in the thermodynamic limit, the replica-symmetric order parameter is a functional. The general theory is developed for arbitrary values of the dimension d of the spins, and arbitrary choices of the ensemble of random orthogonal matrices. We calculate phase diagrams and the values of moments of the order parameter explicitly for d = 2 (finitely connected XY spins with random chiral interactions) and for d = 3 (finitely connected classical Heisenberg spins with random chiral interactions). Numerical simulations are shown to support our predictions quite satisfactorily.

show abstract

“…Moreover, we consider a highly sparse network, with values of κ/ N ∈ [10 −3 , 10 −2 ], which can be homogeneous (i.e., every node having roughly the same connectivity degree), or heterogeneous, with the formation of hubs. Both sparseness and heterogeneity damage severely the memory retrieval ability of the neural network that, for such cases, diminishes fast with P compared with the case of highly connected and homogeneous neural networks (Stauffer et al, 2003; Castillo et al, 2004; Morelli et al, 2004; Torres et al, 2004; Oshima and Odagaki, 2007; Akam and Kullmann, 2014) However, there is experimental evidence that the configurations of neural activity related to particular memories in the animal brain involve many more silent neurons, ξiμ=0, than active ones, ξiμ=1 (Chklovskii et al, 2004; Akam and Kullmann, 2014). Notice that in this case there is a positive correlation between different patterns due to the sparseness, since a 0 ≠ 0.5, which is also known to improve the storage capacity of a neural network (Knoblauch et al, 2014; Knoblauch and Sommer, 2016), and in particular that of heterogeneous and sparse neural networks (Morelli et al, 2004).…”

Section: Model and Methodsmentioning

confidence: 99%

How Memory Conforms to Brain Development

Millán

Torres

Marro

2019

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Nature exhibits countless examples of adaptive networks , whose topology evolves constantly coupled with the activity due to its function. The brain is an illustrative example of a system in which a dynamic complex network develops by the generation and pruning of synaptic contacts between neurons while memories are acquired and consolidated. Here, we consider a recently proposed brain developing model to study how mechanisms responsible for the evolution of brain structure affect and are affected by memory storage processes. Following recent experimental observations, we assume that the basic rules for adding and removing synapses depend on local synaptic currents at the respective neurons in addition to global mechanisms depending on the mean connectivity. In this way a feedback loop between “form” and “function” spontaneously emerges that influences the ability of the system to optimally store and retrieve sensory information in patterns of brain activity or memories. In particular, we report here that, as a consequence of such a feedback-loop, oscillations in the activity of the system among the memorized patterns can occur, depending on parameters, reminding mind dynamical processes. Such oscillations have their origin in the destabilization of memory attractors due to the pruning dynamics, which induces a kind of structural disorder or noise in the system at a long-term scale. This constantly modifies the synaptic disorder induced by the interference among the many patterns of activity memorized in the system. Such new intriguing oscillatory behavior is to be associated only to long-term synaptic mechanisms during the network evolution dynamics, and it does not depend on short-term synaptic processes, as assumed in other studies, that are not present in our model.

show abstract

Analytic solution of attractor neural networks on scale-free graphs

Cited by 35 publications

References 29 publications

Efficient architectures for sparsely-connected high capacity associative memory models

Efficient architectures for sparsely-connected high capacity associative memory models

Finitely connected vector spin systems with random matrix interactions

How Memory Conforms to Brain Development

Contact Info

Product

Resources

About