Low-rank Riemannian eigensolver for high-dimensional Hamiltonians

Rakhuba, Maxim; Novikov, Alexander; Oseledets, Ivan

doi:10.1016/j.jcp.2019.07.003

Cited by 11 publications

(9 citation statements)

References 53 publications

(101 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, R). An 'optimized' version of this operation combines the matrix-by-vector multiplication and the projection onto the tangent space into a single step P X AZ and exploits the structure of arising operations to decrease complexity to O(dn 2 r x r z R 2 ) (for details of implementation of this operation see section 4.1 of [21]).…”

Section: Stop-gradient and A Wider Class Of Functionalsmentioning

confidence: 99%

Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds

Novikov¹,

Rakhuba²,

Oseledets³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. Since matrices and tensors of fixed rank form smooth Riemannian manifolds, one of the popular tools for finding the low-rank approximations is to use the Riemannian optimization. Nevertheless, efficient implementation of Riemannian gradients and Hessians, required in Riemannian optimization algorithms, can be a nontrivial task in practice. Moreover, in some cases, analytic formulas are not even available. In this paper, we build upon automatic differentiation and propose a method that, given an implementation of the function to be minimized, efficiently computes Riemannian gradients and matrix-by-vector products between approximate Riemannian Hessian and a given vector.

show abstract

Section: Stop-gradient and A Wider Class Of Functionalsmentioning

confidence: 99%

Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds

Novikov¹,

Rakhuba²,

Oseledets³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the representation of the model in tensor-train format and the fact, that the given loss function ( 4) is quadratic with respect to the tensor α allow for usage of more productive optimization methods. In this work, we suggest using Riemannian optimization, which is a promising tool for learning tensor-based models [Rakhuba et al, 2019, Steinlechner, 2016.…”

Section: Learning Via Riemannian Optimizationmentioning

confidence: 99%

Tensor-Train Density Estimation

Novikov¹,

Panov²,

Oseledets³

2021

Preprint

View full text Add to dashboard Cite

Estimation of probability density function from samples is one of the central problems in statistics and machine learning. Modern neural networkbased models can learn high dimensional distributions but have problems with hyperparameter selection and are often prone to instabilities during training and inference. We propose a new efficient tensor train-based model for density estimation (TTDE). Such density parametrization allows exact sampling, calculation of cumulative and marginal density functions, and partition function. It also has very intuitive hyperparameters. We develop an efficient non-adversarial training procedure for TTDE based on the Riemannian optimization. Experimental results demonstrate the competitive performance of the proposed method in density estimation and sampling tasks, while TTDE significantly outperforms competitors in training speed.

show abstract

“…In particular, thanks to the multilinear structure of the TT format, it is feasible to minimize globally over a subspace in one of the TT cores. Proceeding in a sweeping manner, one can mimic the Jacob-Davidson method to TT tensors; see [91,92].…”

Section: Eigenvalue Problemsmentioning

confidence: 99%

Geometric Methods on Low-Rank Matrix and Tensor Manifolds

Uschmajew

Vandereycken

2020

Handbook of Variational Methods for Nonlinear Geometric Data

View full text Add to dashboard Cite

In this chapter we present numerical methods for low-rank matrix and tensor problems that explicitly make use of the geometry of rank constrained matrix and tensor spaces. We focus on two types of problems: The first are optimization problems, like matrix and tensor completion, solving linear systems and eigenvalue problems. Such problems can be solved by numerical optimization for manifolds, called Riemannian optimization methods. We will explain the basic elements of differential geometry in order to apply such methods efficiently to rank constrained matrix and tensor spaces. The second type of problem is ordinary differential equations, defined on matrix and tensor spaces. We show how their solution can be approximated by the dynamical low-rank principle, and discuss several numerical integrators that rely in an essential way on geometric properties that are characteristic to sets of low rank matrices and tensors.

show abstract

Low-rank Riemannian eigensolver for high-dimensional Hamiltonians

Cited by 11 publications

References 53 publications

Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds

Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds

Tensor-Train Density Estimation

Geometric Methods on Low-Rank Matrix and Tensor Manifolds

Contact Info

Product

Resources

About