Equilibrium statistical mechanics of bipartite spin systems

Barra, Adriano; Genovese, Giuseppe; Guerra, Francesco

doi:10.1088/1751-8113/44/24/245002

Cited by 77 publications

(118 citation statements)

References 33 publications

Supporting

Mentioning

111

Contrasting

Order By: Relevance

“…When instead the interaction coefficients are on the hyperbolic regime the model is beyond the classical techniques available to solve it. The case K = 2 is known in the litterature as bipartite spin glass model and has been studied in [4,7].…”

Section: Introduction and Resultsmentioning

confidence: 99%

Annealing and Replica-Symmetry in Deep Boltzmann Machines

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this paper we study the properties of the quenched pressure of a multi-layer spin-glass model (a deep Boltzmann Machine in artificial intelligence jargon) whose pairwise interactions are allowed between spins lying in adjacent layers and not inside the same layer nor among layers at distance larger than one. We prove a theorem that bounds the quenched pressure of such a K-layer machine in terms of K Sherrington-Kirkpatrick spin glasses and use it to investigate its annealed region. The replica-symmetric approximation of the quenched pressure is identified and its relation to the annealed one is considered. The paper also presents some observation on the model's architectural structure related to machine learning. Since escaping the annealed region is mandatory for a meaningful training, by squeezing such region we obtain thermodynamical constraints on the form factors. Remarkably, its optimal escape is achieved by requiring the last layer to scale sub-linearly in the network size.

show abstract

Section: Introduction and Resultsmentioning

confidence: 99%

Annealing and Replica-Symmetry in Deep Boltzmann Machines

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…2 should not be expected to directly apply to this more general case. 18 Additionally, it is very difficult to see how the 16 After the first version of this paper appeared, we learned that this same approach was used in [46]. 17 Although the Hamiltonians for both the bipartite SK model and the RBM are of the same form, the standard convention is that the RBM variables take on the values vi, hj ∈ {0, 1}.…”

Section: Application: Restricted Boltzmann Machinesmentioning

confidence: 99%

Replica symmetry breaking in bipartite spin glasses and neural networks

2018

View full text Add to dashboard Cite

Some interesting recent advances in the theoretical understanding of neural networks have been informed by results from the physics of disordered many-body systems. Motivated by these findings, this work uses the replica technique to study the mathematically tractable bipartite Sherrington-Kirkpatrick (SK) spin-glass model, which is formally similar to a restricted Boltzmann machine (RBM) neural network. The bipartite SK model has been previously studied assuming replica symmetry; here this assumption is relaxed and a replica symmetry breaking analysis is performed. The bipartite SK model is found to have many features in common with Parisi's solution of the original, unipartite SK model, including the existence of a multitude of pure states which are related in a hierarchical, ultrametric fashion. As an application of this analysis, the optimal cost for a graph partitioning problem is shown to be simply related to the ground state energy of the bipartite SK model. As a second application, empirical investigations reveal that the Gibbs sampled outputs of an RBM trained on the MNIST data set are more ultrametrically distributed than the input data themselves.

show abstract

“…In order to examine models more faithful to real-world networks, physicists and mathematicians have in recent years advanced the study of bipartite or more general "multi-species" spin systems, e.g. [24,14,22,23,12,11]. An ongoing task is to adapt results from classical spin glasses, such as SK, to their multi-species extensions, especially those regarding the so-called glassy phase observed at low temperatures.…”

Section: Introductionmentioning

confidence: 99%

Replica Symmetry Breaking in Multi-species Sherrington–Kirkpatrick Model

2019

View full text Add to dashboard Cite

In the Sherrington-Kirkpatrick (SK) and related mixed p-spin models, there is interest in understanding replica symmetry breaking at low temperatures. For this reason, the so-called AT line proposed by de Almeida and Thouless as a sufficient (and conjecturally necessary) condition for symmetry breaking, has been a frequent object of study in spin glass theory. In this paper, we consider the analogous condition for the multi-species SK model, which concerns the eigenvectors of a Hessian matrix. The analysis is tractable in the two-species case with positive definite variance structure, for which we derive an explicit AT temperature threshold. To our knowledge, this is the first nonasymptotic symmetry breaking condition produced for a multi-species spin glass. As possible evidence that the condition is sharp, we draw further parallel with the classical SK model and show coincidence with a separate temperature inequality guaranteeing uniqueness of the replica symmetric critical point.2010 Mathematics Subject Classification. 60K35, 82B26, 82B44.

show abstract

Equilibrium statistical mechanics of bipartite spin systems

Cited by 77 publications

References 33 publications

Annealing and Replica-Symmetry in Deep Boltzmann Machines

Annealing and Replica-Symmetry in Deep Boltzmann Machines

Replica symmetry breaking in bipartite spin glasses and neural networks

Replica Symmetry Breaking in Multi-species Sherrington–Kirkpatrick Model

Contact Info

Product

Resources

About