The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems

Abstract

Autoregressive Neural Networks (ARNNs) have shown exceptional results in generation tasks across image, language, and scientific domains. Despite their success, ARNN architectures often operate as black boxes without a clear connection to underlying physics or statistical models. This research derives an exact mapping of the Boltzmann distribution of binary pairwise interacting systems in autoregressive form. The parameters of the ARNN are directly related to the Hamiltonian’s couplings and external fields, and commonly used structures like residual connections and recurrent architecture emerge from the derivation. This explicit formulation leverages statistical physics techniques to derive ARNNs for specific systems. Using the Curie–Weiss and Sherrington–Kirkpatrick models as examples, the proposed architectures show superior performance in replicating the associated Boltzmann distributions compared to commonly used designs. The findings foster a deeper connection between physical systems and neural network design, paving the way for tailored architectures and providing a physical lens to interpret existing ones.

Sparse autoregressive neural networks for classical spin systems

“…The ARNN architectures used are typically adapted from those developed for data-driven problems in computer science, and prior knowledge on theoretical properties of the physical system has rarely been utilized to customize the ARNN architecture, or only been exploited for specific physical systems (Biał as et al 2022, Biazzo et al 2022). Recently, a direct mapping was established between the Boltzmann distribution and the corresponding ARNN architecture (Biazzo 2023). The derived general architecture was not directly usable, but thanks to the analytic derivation, two specific architectures for two well-known fully connected spin models, the Curie-Weiss model and the Sherringhton-Kirkpatrick model, were derived.…”

“…In Biazzo (2023), the functions ρ i were approximated with specific architectures derived for the fully connected Curie-Weiss and Sherrington-Kirkpatrick models. In the present work, we propose a more general approach to approximate ρ i using feedforward neural networks, whose inputs are the variables ξ i , and to take advantage of the sparsity of interactions among the spins.…”