Iterative Methods Based on Soft Thresholding of Hierarchical Tensors

Abstract

We construct a soft thresholding operation for rank reduction in hierarchical tensors and subsequently consider its use in iterative thresholding methods, in particular for the solution of discretized high-dimensional elliptic problems. The proposed method for the latter case adjusts the thresholding parameters, by an a posteriori criterion requiring only bounds on the spectrum of the operator, such that the arising tensor ranks of the resulting iterates remain quasi-optimal with respect to the algebraic or exponential-type decay of the hierarchical singular values of the true solution. In addition, we give a modified algorithm using inexactly evaluated residuals that retains these features. The effectiveness of the scheme is demonstrated in numerical experiments.Keywords Low-rank tensor approximation · Hierarchical tensor format · Soft thresholding · High-dimensional elliptic problems Mathematics Subject Classification 41A46 · 41A63 · 65D99 · 65F10 · 65N12 · 65N15 Communicated by Endre Süli.

Stability of Low-Rank Tensor Representations and Structured Multilevel Preconditioning for Elliptic PDEs

“…In the present paper, by analytically combining the low-rank representations of the preconditioner and of the stiffness matrix, we obtain a tensor representation that retains favorable representation condition numbers also for large L and leads to solvers that remain numerically stable even for h on the order of the machine precision . For the problems preconditioned in this manner, we can apply results from [4,6] to obtain bounds for the number of operations required for computing u LR h , in terms of the ranks of low-rank best approximations of u h with the same error. Since the costs depend only weakly on the discretization level L, one may then in fact simply choose L so large that h ≈ .…”

“…We prove a new result on a BPX preconditioner for second-order elliptic problems that is tailored to our purposes, and we construct a low-rank decomposition of the preconditioned stiffness matrix with the following properties: it is well-conditioned uniformly in discretization level L as a matrix; its ranks are independent of L; and its representation condition numbers remain moderate for large L. Based on these properties, we establish an estimate for the total computational complexity of finding approximate solutions in low-rank form. These complexity bounds are shown for an iterative solver based on the soft thresholding of tensors [6], for which the ranks of approximate solutions can be estimated in terms of the ranks of the exact Galerkin solution. We identify appropriate approximability assumptions on solutions in the present context, which are slightly different from those proved in [34].…”

“…This construction directly carries over to the special case of the TT format, leading to a soft thresholding operation S α that is non-expansive with respect to the 2 -norm. It can be realized numerically for TT representations, described in [6,Sec. 3], at essentially the same cost as the TT-SVD.…”

“…In cases with variable coefficients such that B L does not have an exact low-rank form, but needs to be applied approximately, the iteration given in (6.1) and (6.2) can be adapted to residual approximations with prescribed tolerance as given in [6,Alg. 3], which preserves the statement of Theorem 6.3 as shown in [6,Prop. 5.9].…”

Approximation of Singularities by Quantized‐Tensor FEM

“…O(l θ ) with a small θ ≥ 1. This property is crucial for the applicability of tensor-structured methods; we refer to the papers [14][15][16][17][18][19][20][21][22][23][24], to the literature survey [25] and more recent works [26][27][28].…”