Randomized algorithms for fast computation of low rank tensor ring model

Ahmadi‐Asl, Salman; Cichocki, Andrzej; Phan, Anh Huy; Asante-Mensah, Maame G.; Ghazani, Mirfarid Musavian; Tanaka, Toshihisa; Oseledets, Ivan

doi:10.1088/2632-2153/abad87

Cited by 14 publications

(12 citation statements)

References 102 publications

(46 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another advantage of the STHSOVD compared to the THOSVD is that in the STHOSVD algorithm, the core tensor is automatically computed by the algorithm in the last step while in the THOSVD algorithm, all factor matrices are first computed and then the core tensor is computed through formulation (17) or (19). This may lead to the intermediate data explosion phenomenon [15].…”

Section: A Sequentially Truncated Hosvd (Sthosvd) Algorithmmentioning

confidence: 99%

“…• Maxvol-based low-rank matrix approximation [98], [99] • Cross2D matrix approximation [100], [101] • Discrete Empirical Interpolatory Method (DEIM) [103], [104] • Pivoted QR decomposition [105] It is know that the quality of the cross approximation quite depends on the module of the determinant of the intersection matrix 19 which is called matrix volume. More precisely, a set of columns and rows should be selected with an intersection matrix whose volume is as much as possible.…”

Section: B Randomized Sampling Tucker Decompositionmentioning

confidence: 99%

“…Another possible direction is using the count-sketch technique for low-rank matrix approximations of the unfolding matrices which are required for tensor decomposition. Here, the factor matrices are approximated by computing low-rank approximation of the unfolding matrices using the countsketch technique and then the core tensor is computed via (19).…”

Section: Randomized Count-sketch Tucker Decompositionmentioning

confidence: 99%

“…Deterministic algorithms for decomposing large-scale data tensors into the Tucker format are prohibitive and require high memory and computational complexity or only applicable for structured data tensors [15], [16]. It has been proved that randomized algorithms can tackle this difficulty by exploiting only a part of the data tensors with applications in tensor regression [17], tensor completion [18], [19] and deep learning [20]. Because of this property, they scale quite well to the tensor dimensions and orders.…”

Section: Introductionmentioning

confidence: 99%

“…It is also called (pseudo)-skeleton decomposition or CUR decomposition 19. By an intersection matrix, we mean a matrix which produced by the intersection of sampled columns and rows.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

et al. 2021

Self Cite

View full text Add to dashboard Cite

Big data analysis has become a crucial part of new emerging technologies such as the internet of things, cyber-physical analysis, deep learning, anomaly detection, etc. Among many other techniques, dimensionality reduction plays a key role in such analyses and facilitates feature selection and feature extraction. Randomized algorithms are efficient tools for handling big data tensors. They accelerate decomposing large-scale data tensors by reducing the computational complexity of deterministic algorithms and the communication among different levels of memory hierarchy, which is the main bottleneck in modern computing environments and architectures. In this paper, we review recent advances in randomization for computation of Tucker decomposition and Higher Order SVD (HOSVD). We discuss random projection and sampling approaches, single-pass and multi-pass randomized algorithms and how to utilize them in the computation of the Tucker decomposition and the HOSVD. Simulations on synthetic and real datasets are provided to compare the performance of some of best and most promising algorithms.

show abstract

Section: A Sequentially Truncated Hosvd (Sthosvd) Algorithmmentioning

confidence: 99%

Section: B Randomized Sampling Tucker Decompositionmentioning

confidence: 99%

Section: Randomized Count-sketch Tucker Decompositionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…It is also called (pseudo)-skeleton decomposition or CUR decomposition 19. By an intersection matrix, we mean a matrix which produced by the intersection of sampled columns and rows.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

Practical alternating least squares for tensor ring decomposition

Yu,

2023

Numerical Linear Algebra App

View full text Add to dashboard Cite

Tensor ring (TR) decomposition has been widely applied as an effective approach in a variety of applications to discover the hidden low‐rank patterns in multidimensional and higher‐order data. A well‐known method for TR decomposition is the alternating least squares (ALS). However, solving the ALS subproblems often suffers from high cost issue, especially for large‐scale tensors. In this paper, we provide two strategies to tackle this issue and design three ALS‐based algorithms. Specifically, the first strategy is used to simplify the calculation of the coefficient matrices of the normal equations for the ALS subproblems, which takes full advantage of the structure of the coefficient matrices of the subproblems and hence makes the corresponding algorithm perform much better than the regular ALS method in terms of computing time. The second strategy is to stabilize the ALS subproblems by QR factorizations on TR‐cores, and hence the corresponding algorithms are more numerically stable compared with our first algorithm. Extensive numerical experiments on synthetic and real data are given to illustrate and confirm the above results. In addition, we also present the complexity analyses of the proposed algorithms.

show abstract

Practical sketching‐based randomized tensor ring decomposition

Yu,

2024

Numerical Linear Algebra App

View full text Add to dashboard Cite

Based on sketching techniques, we propose two practical randomized algorithms for tensor ring (TR) decomposition. Specifically, on the basis of defining new tensor products and investigating their properties, the two algorithms are devised by applying the Kronecker sub‐sampled randomized Fourier transform and TensorSketch to the alternating least squares subproblems derived from the minimization problem of TR decomposition. From the former, we find an algorithmic framework based on random projection for randomized TR decomposition. We compare our proposals with the existing methods using both synthetic and real data. Numerical results show that they have quite decent performance in accuracy and computing time.

show abstract

Randomized algorithms for fast computation of low rank tensor ring model

Cited by 14 publications

References 102 publications

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

Practical alternating least squares for tensor ring decomposition

Practical sketching‐based randomized tensor ring decomposition

Contact Info

Product

Resources

About