Capacity of an associative memory model on random graph architectures

Abstract

We analyze the storage capacity of the Hopfield models on classes of random graphs. While such a setup has been analyzed for the case that the underlying random graph model is an Erd\"{o}s-Renyi graph, other architectures, including those investigated in the recent neuroscience literature, have not been studied yet. We develop a notion of storage capacity that highlights the influence of the graph topology and give results on the storage capacity for not too irregular random graph models. The class of models investigated includes the popular power law graphs for some parameter values.Comment: Published at http://dx.doi.org/10.3150/14-BEJ630 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

Spatially Arranged Sparse Recurrent Neural Networks for Energy Efficient Associative Memory

“…This early study was followed by the analyses of the associative memory ability of randomly diluted Hopfield networks [14]- [19]. In addition, much attention has been paid to the Hopfield networks with complex topologies including small-world networks [20]- [22], scale-free networks [23], and other architectures [24], [25]. Some studies suggest that homogeneously connected random networks are better than heterogeneously connected ones in the storage capacity when the interconnection density is the same [20], [26], [27].…”

A Comparative Study of Sparse Associative Memories

“…These probabilities are notoriously difficult to access analytically (see e.g. [5], [18], or [19]). The simulations give an impression of the advantages and drawbacks of the several models.…”

Bounding Extremal Degrees of Edge-Independent Random Graphs Using Relative Entropy

“…This result is already known (see, e.g., p.152 in [2]) and is proved by a more sophisticated method called Stein's method. A more or less similar approach appears in [21].…”

“…However, this result is already known to be true under an even weaker condition, namely, np ln n (see, e.g., p.72, Cor. 3.14 in [1], [21]). It is viable to expect that our Theorem 1 holds as long as ∆(A) ln n. Unfortunately, we do not have a proof presently.…”