The Replica Symmetric Approximation of the Analogical Neural Network

Barra, Adriano; Genovese, Giuseppe; Guerra, Francesco

doi:10.1007/s10955-010-0020-y

Cited by 58 publications

(126 citation statements)

References 15 publications

Supporting

Mentioning

123

Contrasting

Order By: Relevance

“…From a theoretical physics perspective, a network of interacting B-and T-cells resembles a bi-partite spin glass. It was recently shown that such a bi-partite spin-glass is thermodynamically equivalent to a Hopfield-like neural network with effective Hebbian interactions [20,21].…”

Section: Introductionmentioning

confidence: 99%

Immune networks: multitasking capabilities near saturation

Agliari

Annibale

Barra

et al. 2013

J. Phys. A: Math. Theor.

120

View full text Add to dashboard Cite

Abstract. Pattern-diluted associative networks were introduced recently as models for the immune system, with nodes representing T-lymphocytes and stored patterns representing signalling protocols between T-and B-lymphocytes. It was shown earlier that in the regime of extreme pattern dilution, a system with N T Tlymphocytes can manage a number N B = O(N δ T ) of B-lymphocytes simultaneously, with δ < 1. Here we study this model in the extensive load regime N B = αN T , with also a high degree of pattern dilution, in agreement with immunological findings. We use graph theory and statistical mechanical analysis based on replica methods to show that in the finite-connectivity regime, where each T-lymphocyte interacts with a finite number of B-lymphocytes as N T → ∞, the T-lymphocytes can coordinate effective immune responses to an extensive number of distinct antigen invasions in parallel. As α increases, the system eventually undergoes a second order transition to a phase with clonal cross-talk interference, where the system's performance degrades gracefully. Mathematically, the model is equivalent to a spin system on a finitely connected graph with many short loops, so one would expect the available analytical methods, which all assume locally tree-like graphs, to fail. Yet it turns out to be solvable. Our results are supported by numerical simulations.

show abstract

Section: Introductionmentioning

confidence: 99%

Immune networks: multitasking capabilities near saturation

Agliari

Annibale

Barra

et al. 2013

J. Phys. A: Math. Theor.

120

View full text Add to dashboard Cite

show abstract

“…Moreover, we consider the equivalence between the Hopfield model and a class of Boltzmann machines (Bengio, 2009) developed in Barra, Bernacchia, Santucci, and Contucci (2012) and Barra, Guerra, and Genovese (2010) and we show that this equivalence is rather robust and can be established also for the correlated and diluted Hopfield studied here. Interestingly, this approach allows the investigation of dynamic properties of the model which are as well discussed.…”

Section: Introductionmentioning

confidence: 76%

Parallel retrieval of correlated patterns: From Hopfield networks to Boltzmann machines

et al. 2013

Self Cite

View full text Add to dashboard Cite

a b s t r a c tIn this work, we first revise some extensions of the standard Hopfield model in the low storage limit, namely the correlated attractor case and the multitasking case recently introduced by the authors. The former case is based on a modification of the Hebbian prescription, which induces a coupling between consecutive patterns and this effect is tuned by a parameter a. In the latter case, dilution is introduced in pattern entries, in such a way that a fraction d of them is blank. Then, we merge these two extensions to obtain a system able to retrieve several patterns in parallel and the quality of retrieval, encoded by the set of Mattis magnetizations {m µ }, is reminiscent of the correlation among patterns. By tuning the parameters d and a, qualitatively different outputs emerge, ranging from highly hierarchical to symmetric. The investigations are accomplished by means of both numerical simulations and statistical mechanics analysis, properly adapting a novel technique originally developed for spin glasses, i.e. the Hamilton-Jacobi interpolation, with excellent agreement. Finally, we show the thermodynamical equivalence of this associative network with a (restricted) Boltzmann machine and study its stochastic dynamics to obtain even a dynamical picture, perfectly consistent with the static scenario earlier discussed.

show abstract

“…We conjecture this is a consequence of the choice of the decomposition (2.3). Finally, with the aim of promoting cross-fertilization among the two disciplines of Machine Learning and Disordered Statistical Mechanics, we collected the outlined results by using two among the most used methods to deal with spin-glasses, namely the replica trick [19] and the interpolation method [12], discussing both of them in great detail.…”

Section: Discussionmentioning

confidence: 99%

Interpolating between boolean and extremely high noisy patterns through minimal dense associative memories

Alemanno¹,

Centonze²,

Fachechi³

2020

J. Phys. A: Math. Theor.

View full text Add to dashboard Cite

Recently, Hopfield and Krotov introduced the concept of dense associative memories [DAM] (close to spin-glasses with P -wise interactions in a disordered statistical mechanical jargon): they proved a number of remarkable features these networks share and suggested their use to (partially) explain the success of the new generation of Artificial Intelligence. Thanks to a remarkable ante-litteram analysis by Baldi & Venkatesh, among these properties, it is known these networks can handle a maximal amount of stored patterns K scaling as K ∼ N P −1 . In this paper, once introduced a minimal dense associative network as one of the most elementary cost-functions falling in this class of DAM, we sacrifice this high-load regime -namely we force the storage of solely a linear amount of patterns, i.e. K = αN (with α > 0)-to prove that, in this regime, these networks can correctly perform pattern recognition even if pattern signal is O(1) and is embedded in a sea of noise O( √ N ), also in the large N limit. To prove this statement, by extremizing the quenched free-energy of the model over its natural order-parameters (the various magnetizations and overlaps), we derived its phase diagram, at the replica symmetric level of description and in the thermodynamic limit: as a sideline, we stress that, to achieve this task, aiming at cross-fertilization among disciplines, we pave two hegemon routes in the statistical mechanics of spin glasses, namely the replica trick and the interpolation technique. Both the approaches reach the same conclusion: there is a not-empty region, in the noise-T vs load-α phase diagram plane, where these networks can actually work in this challenging regime; in particular we obtained a quite high critical (linear) load in the (fast) noiseless case resulting in lim β→∞ α c (β) = 0.65. arXiv:1912.00666v1 [cond-mat.dis-nn]

show abstract

The Replica Symmetric Approximation of the Analogical Neural Network

Cited by 58 publications

References 15 publications

Immune networks: multitasking capabilities near saturation

Immune networks: multitasking capabilities near saturation

Parallel retrieval of correlated patterns: From Hopfield networks to Boltzmann machines

Interpolating between boolean and extremely high noisy patterns through minimal dense associative memories

Contact Info

Product

Resources

About