We propose a general framework for solving statistical mechanics of systems with finite size. The approach extends the celebrated variational mean-field approaches using autoregressive neural networks, which support direct sampling and exact calculation of normalized probability of configurations. It computes variational free energy, estimates physical quantities such as entropy, magnetizations and correlations, and generates uncorrelated samples all at once. Training of the network employs the policy gradient approach in reinforcement learning, which unbiasedly estimates the gradient of variational parameters. We apply our approach to several classic systems, including 2D Ising models, the Hopfield model, the Sherrington-Kirkpatrick model, and the inverse Ising model, for demonstrating its advantages over existing variational mean-field methods. Our approach sheds light on solving statistical physics problems using modern deep generative neural networks.Consider a statistical physics model such as the celebrated Ising model, the joint probability of spins s ∈ {±1} N follows the Boltzmann distributionwhere β = 1/T is the inverse temperature and Z is the partition function. Given a problem instance, statistical mechanics problems concern about how to estimate the free energy F = − 1 β ln Z of the instance, how to compute macroscopic properties of the system such as magnetizations and correlations, and how to sample from the Boltzmann distribution efficiently. Solving these problems are not only relevant to physics, but also find broad applications in fields like Bayesian inference where the Boltzmann distribution naturally acts as posterior distribution, and in combinatorial optimizations where the task is equivalent to study zero temperature phase of a spin-glass model.When the system has finite size, computing exactly the free energy belongs to the class of #P-hard problems, hence is in general intractable. Therefore, usually one employs approximate algorithms such as variational approaches. The variational approach adopts an ansatz for the joint distribution q θ (s) parametrized by variational parameters θ, and adjusts them so that q θ (s) is as close as possible to the Boltzmann distribution p(s). The closeness between two distributions is measured bywhereis the variational free energy corresponding to distribution q θ (s). Since the KL divergence is non-negative, minimizing the KL divergence is equivalent to minimizing the variational free energy F q , an upper bound to the true free energy F.One of the most popular variational approaches, namely the variational mean-field method, assumes a factorized vari-is the marginal probability of the ith spin. In such parametrization, the variational free energy F q can be expressed as an analytical function of parameters q i (s i ), as well as its derivative with respect to q i (s i ). By setting the derivatives to zero, one obtains a set of iterative equations, known as the naïve mean-field (NMF) equations. Despite its simplicity, NMF has been used in various applica...
We introduce version 3 of NetKet, the machine learning toolbox for many-body quantum physics. NetKet is built around neural quantum states and provides efficient algorithms for their evaluation and optimization. This new version is built on top of JAX, a differentiable programming and accelerated linear algebra framework for the Python programming language. The most significant new feature is the possibility to define arbitrary neural network ansätze in pure Python code using the concise notation of machine-learning frameworks, which allows for just-in-time compilation as well as the implicit generation of gradients thanks to automatic differentiation. NetKet 3 also comes with support for GPU and TPU accelerators, advanced support for discrete symmetry groups, chunking to scale up to thousands of degrees of freedom, drivers for quantum dynamics applications, and improved modularity, allowing users to use only parts of the toolbox as a foundation for their own code.
Flow-based generative models have become an important class of unsupervised learning approaches. In this work, we incorporate the key ideas of renormalization group (RG) and sparse prior distribution to design a hierarchical flow-based generative model, RG-Flow, which can separate information at different scales of images and extract disentangled representations at each scale. We demonstrate our method on synthetic multi-scale image datasets and the CelebA dataset, showing that the disentangled representations enable semantic manipulation and style mixing of the images at different scales. To visualize the latent representations, we introduce receptive fields for flow-based models and show that the receptive fields of RG-Flow are similar to those of convolutional neural networks. In addition, we replace the widely adopted isotropic Gaussian prior distribution by the sparse Laplacian distribution to further enhance the disentanglement of representations. From a theoretical perspective, our proposed method has $O(\log L)$ complexity for inpainting of an image with edge length $L$, compared to previous generative models with $O(L^2)$ complexity.
We introduce version 3 of NetKet, the machine learning toolbox for many-body quantum physics. NetKet is built around neural quantum states and provides efficient algorithms for their evaluation and optimization. This new version is built on top of JAX, a differentiable programming and accelerated linear algebra framework for the Python programming language. The most significant new feature is the possibility to define arbitrary neural network ansätze in pure Python code using the concise notation of machine-learning frameworks, which allows for just-in-time compilation as well as the implicit generation of gradients thanks to automatic differentiation. NetKet 3 also comes with support for GPU and TPU accelerators, advanced support for discrete symmetry groups, chunking to scale up to thousands of degrees of freedom, drivers for quantum dynamics applications, and improved modularity, allowing users to use only parts of the toolbox as a foundation for their own code.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.