A Riemannian approach to low-rank tensor learning

Kasai, Hiroyuki; Jawanpuria, Pratik; Mishra, Bamdev

doi:10.1016/b978-0-12-824447-0.00010-8

Cited by 10 publications

(23 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For previous work on SGD on Riemannian manifolds, we refer the reader to [18][19][20][21][22]. The presented technique is completely intrinsic to the manifold N and involves following (approximate) geodesics in the direction of the (negative) gradient of L. To explain this idea in more detail, we first briefly recall the notion of geodesics and refer the reader to [27,28] for a more comprehensive introduction to differential geometry.…”

Section: Stochastic Gradient Descentmentioning

confidence: 99%

“…In particular, we explain in detail how to perform SGD on Riemannian manifolds arising from a finite-dimensional system of equations. Performing SGD on Riemannian manifolds has been studied before, e.g., [18][19][20][21][22]. Our method, in particular, heavily relies on the Implicit Function Theorem, which is used to construct explicit charts amenable to numerical computations.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Constrained Empirical Risk Minimization: Theory and Practice

Marcus¹,

Sheombarsing²,

Sonke³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) are widely used for their ability to effectively approximate large classes of functions. This flexibility, however, makes the strict enforcement of constraints on DNNs an open problem. Here we present a framework that, under mild assumptions, allows the exact enforcement of constraints on parameterized sets of functions such as DNNs. Instead of imposing "soft" constraints via additional terms in the loss, we restrict (a subset of) the DNN parameters to a submanifold on which the constraints are satisfied exactly throughout the entire training procedure. We focus on constraints that are outside the scope of equivariant networks used in Geometric Deep Learning. As a major example of the framework, we restrict filters of a Convolutional Neural Network (CNN) to be wavelets, and apply these wavelet networks to the task of contour prediction in the medical domain.

show abstract

Section: Stochastic Gradient Descentmentioning

confidence: 99%

mentioning

confidence: 99%

Constrained Empirical Risk Minimization: Theory and Practice

Marcus¹,

Sheombarsing²,

Sonke³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The construction of this metric aims to approximate the Hessian of the cost function by its "diagonal blocks". These algorithms improve the performance of Euclidean-based algorithms, and are successfully applied to matrix and tensor completion (e.g., [25,19,12,9]). However, the extension to TR is not straightforward since it involves large matrix computation and formulation.…”

Section: Preliminariesmentioning

confidence: 99%

“…Therefore, low-rank matrix decompositions are widely used in matrix completion, which can save the computational cost and storage. In the same spirit, lowrank tensor decompositions play a significant role in tensor completion; applications can be found across various fields, e.g., recommendation systems [19,12], image processing [24], and interpolation of high-dimensional functions [32,21].…”

mentioning

confidence: 99%

“…For tensor completion in Tucker decomposition, Kressner et al [23] proposed a Riemannian conjugate gradient method on the manifold of rank-constrained tensors. Kasai and Mishra [19] introduced a preconditioned metric and considered the quotient geometry of the manifold via Tucker decomposition. The corresponding Riemannian conjugate gradient algorithm was proposed.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Riemannian preconditioned algorithms for tensor completion via tensor ring decomposition

Gao¹,

Peng²,

Yuan³

2023

Preprint

View full text Add to dashboard Cite

We propose Riemannian preconditioned algorithms for the tensor completion problem via tensor ring decomposition. A new Riemannian metric is developed on the product space of the mode-2 unfolding matrices of the core tensors in tensor ring decomposition. The construction of this metric aims to approximate the Hessian of the cost function by its diagonal blocks, paving the way for various Riemannian optimization methods. Specifically, we propose the Riemannian gradient descent and Riemannian conjugate gradient algorithms. We prove that both algorithms globally converge to a stationary point. In the implementation, we exploit the tensor structure and design an economical procedure to avoid large matrix formulation and computation in gradients, which significantly reduces the computational cost. Numerical experiments on various synthetic and real-world datasets-movie ratings, hyperspectral images, and high-dimensional functions-suggest that the proposed algorithms are more efficient and have better reconstruction ability than other candidates.

show abstract

Riemannian block SPD coupling manifold and its application to optimal transport

et al. 2022

View full text Add to dashboard Cite

In this work, we study the optimal transport (OT) problem between symmetric positive definite (SPD) matrix-valued measures. We formulate the above as a generalized optimal transport problem where the cost, the marginals, and the coupling are represented as block matrices and each component block is a SPD matrix. The summation of row blocks and column blocks in the coupling matrix are constrained by the given block-SPD marginals. We endow the set of such block-coupling matrices with a novel Riemannian manifold structure. This allows to exploit the versatile Riemannian optimization framework to solve generic SPD matrix-valued OT problems. We illustrate the usefulness of the proposed approach in several applications.

show abstract

A Riemannian approach to low-rank tensor learning

Cited by 10 publications

References 24 publications

Constrained Empirical Risk Minimization: Theory and Practice

Constrained Empirical Risk Minimization: Theory and Practice

Riemannian preconditioned algorithms for tensor completion via tensor ring decomposition

Riemannian block SPD coupling manifold and its application to optimal transport

Contact Info

Product

Resources

About