Probabilistic Modeling with Matrix Product States

Stokes, James; Terilla, John

doi:10.3390/e21121236

Cited by 30 publications

(24 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typical neural network architectures do not involve sophisticated linear algebra operations. However, with the development of tensorized neural networks [101] and applications of various tensor networks to machine learning problems [10][11][12][13][14]102], the boundary between the two classes of networks is blurred. Thus, results presented this paper would also be relevant to tensor network machine learning applications when one moves to more sophisticated contraction schemes.…”

Section: Discussionmentioning

confidence: 99%

“…Tensor networks are prominent approaches for studying classical statistical physics and quantum many-body physics problems [1][2][3]. In recent years, its application has expanded rapidly to diverse regions include simulating and designing of quantum circuits [4][5][6][7], quantum error correction [8,9], machine learning [10][11][12][13][14], language modeling [15,16], quantum field theory [17][18][19][20] and holography duality [21,22].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Differentiable Programming Tensor Networks

et al. 2019

View full text Add to dashboard Cite

Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and trains them using automatic differentiation. The concept emerges from deep learning but is not only limited to training neural networks. We present theory and practice of programming tensor network algorithms in a fully differentiable way. By formulating the tensor network algorithm as a computation graph, one can compute higher order derivatives of the program accurately and efficiently using automatic differentiation. We present essential techniques to differentiate through the tensor networks contraction algorithms, including numerical stable differentiation for tensor decompositions and efficient backpropagation through fixed point iterations. As a demonstration, we compute the specific heat of the Ising model directly by taking the second order derivative of the free energy obtained in the tensor renormalization group calculation. Next, we perform gradient based variational optimization of infinite projected entangled pair states for quantum antiferromagnetic Heisenberg model and obtain start-of-the-art variational energy and magnetization with moderate efforts. Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications. * wanglei@iphy.ac.cn † txiang@iphy.ac.cn This fact has limited broad adoption of the gradient based optimization of tensor network states to more complex systems. Alternative approaches, such as computing the gradient using numerical derivative has limited accuracy and efficiency, therefore only applies to cases with few variational parameters [36,37]. While deriving the gradient manually using the chain rule is only manageable for purposely designed simple tensor network structures [38].Differentiable programming provides an elegant and efficient solution to these problems by composing the whole tensor network program in a fully differentiable manner. In this paper, we present essential automatic differentiation techniques which compute (higher order) derivatives of tensor network programs efficiently to numeric precision. This progress opens the door to gradient-based (and even Hessian-based) optimization of tensor network states in a general setting. Moreover, computing (higher order) gradients of the output of a tensor network algorithm offer a straightforward approach to compute physical quantities such as the specific heat and magnetic susceptibilities. The differentiable programming approach is agnostic to the detailed lattice geometry, Hamiltonian, and tensor network contraction schemes. Therefore, the approach is general enough to support a wide class of tensor network applications.We will focus on applications which involve two dimensional infinite tensor networks where the differentiable programming techniques offer significant benefits compared to the conventional approaches. We show that after solving m...

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Differentiable Programming Tensor Networks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Once the TNR has been constructed and optimized, the scaling dimensions can also be extracted easily. Here we use the transfer matrix technique [69] to compute the scaling dimensions with the critical inverse temperature β c = ln (1 + √ 2)/2 shown in figure 19. We can see that with the TNR and transfer matrix technique the computed scaling dimensions have accurate results even at quite higher orders.…”

Section: Scaling Dimensionsmentioning

confidence: 96%

“…Tensor network has been a powerful tool to study quantum many-body systems and classical statistical-mechanical models both theoretically [1,2] and numerically [3][4][5][6][7][8]. And recently, it has been proposed as an alternative tool for (quantum) machine learning tasks, both for supervised learning [9][10][11][12][13][14][15][16], and unsupervised learning [17][18][19][20][21][22]. In the family of tensor networks, the multi-scale entanglement renormalization ansatz (MERA) and tensor network renormalization (TNR) are important tensor networks which are inspired by the idea of renormalization group (RG).…”

Section: Introductionmentioning

confidence: 99%

Differentiable programming of isometric tensor networks

Geng

Zou

2022

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Differentiable programming is a new programming paradigm which enables large scale optimization through automatic calculation of gradients also known as auto-differentiation. This concept emerges from deep learning, and has also been generalized to tensor network optimizations. Here, we extend the differentiable programming to tensor networks with isometric constraints with applications to multiscale entanglement renormalization ansatz (MERA) and tensor network renormalization (TNR). By introducing several gradient-based optimization methods for the isometric tensor network and comparing with Evenbly-Vidal method, we show that auto-differentiation has a better performance for both stability and accuracy. We numerically tested our methods on 1D critical quantum Ising spin chain and 2D classical Ising model. We calculate the ground state energy for the 1D quantum model and internal energy for the classical model, and scaling dimensions of scaling operators and find they all agree with the theory well.

show abstract

“…Tensor networks have been successfully applied to several learning tasks including dimensionality reduction [250], unsupervised learning and generative modelling using matrix product states [251][252][253], representation learning with multi-scale tensor networks [254], sequence-to-sequence learning using matrix product operators [255], language modelling [256,257], Bayesian inference [258]. Ref.…”

Section: Quantum Physics-inspired Machine Learningmentioning

confidence: 99%

Machine learning for quantum matter

Carrasquilla

2020

Advances in Physics: X

159

111

View full text Add to dashboard Cite

Quantum matter, the research field studying phases of matter whose properties are intrinsically quantum mechanical, draws from areas as diverse as hard condensed matter physics, materials science, statistical mechanics, quantum information, quantum gravity, and large-scale numerical simulations. Recently, researchers interested in quantum matter and strongly correlated quantum systems have turned their attention to the algorithms underlying modern machine learning with an eye on making progress in their fields. Here we provide a short review on the recent development and adaptation of machine learning ideas for the purpose advancing research in quantum matter, including ideas ranging from algorithms that recognize conventional and topological states of matter in synthetic experimental data, to representations of quantum states in terms of neural networks and their applications to the simulation and control of quantum systems. We discuss the outlook for future developments in areas at the intersection between machine learning and quantum many-body physics.

show abstract

Probabilistic Modeling with Matrix Product States

Cited by 30 publications

References 17 publications

Differentiable Programming Tensor Networks

Differentiable Programming Tensor Networks

Differentiable programming of isometric tensor networks

Machine learning for quantum matter

Contact Info

Product

Resources

About