Replica symmetry breaking in bipartite spin glasses and neural networks

Abstract

Some interesting recent advances in the theoretical understanding of neural networks have been informed by results from the physics of disordered many-body systems. Motivated by these findings, this work uses the replica technique to study the mathematically tractable bipartite Sherrington-Kirkpatrick (SK) spin-glass model, which is formally similar to a restricted Boltzmann machine (RBM) neural network. The bipartite SK model has been previously studied assuming replica symmetry; here this assumption is relaxed and a replica symmetry breaking analysis is performed. The bipartite SK model is found to have many features in common with Parisi's solution of the original, unipartite SK model, including the existence of a multitude of pure states which are related in a hierarchical, ultrametric fashion. As an application of this analysis, the optimal cost for a graph partitioning problem is shown to be simply related to the ground state energy of the bipartite SK model. As a second application, empirical investigations reveal that the Gibbs sampled outputs of an RBM trained on the MNIST data set are more ultrametrically distributed than the input data themselves.

Perovskite neural trees

“…A unique combination of atomic-level control over proton migration in nickelates under high speed e-fields coupled with their ultra-sensitivity to Ni-site orbital occupancy is responsible for generation of the tree-like memory. Our experimentally measured trees can be classified according to number theory as ultrametric (see Supplement Note 2 for a mathematical description) and represent physical realization of a mathematical concept considered critical for solving a myriad of problems in neural computing 26 .…”

“…Low temperature magnetic states found in spin glasses such as CuMn arising from heating-cooling cycles in the 1-20 K temperature range have provided an experimental context 24,25 . Exploring the vast potential of tree-like states in several areas of neuromorphic computing continues to be an intensively studied topic in neural network theory 26,27 .…”

Message-passing Algorithm of Quantum Annealing with Nonstoquastic Hamiltonian

“…(56), (57), (58) and (59) leads to a natural generalization of the results of the previous study. 52) We emphasize that our results include the quantum fluctuation expressed by the transverse field and beyond. In the restricted Boltzmann machine, we need many samples to estimate the expectation and variance in learning.…”

Parallel computing of Edwards–Anderson model

“…Модель спинового стекла используется в таксономии, задачах классификаций, теории информации, биологии, биоиформатики, свертывании и замораживании белков [3], в нейронных сетях, в оценках эффективности работы квантовых компьютеров, в задачах оптимизации. В процессе развития теории нейронных сетей, в частности, после создания сети Хопфилда [4][5][6], ограниченной машины Больцмана [7], глубокой машины Больцмана [8] интерес к спиновым стеклам значительно усилился. В сети Хопфилда поиск образа по искаженному входному сводится к задаче поиска метастабильных состояний с максимальной корреляцией входного образа с обучаемым образом.…”