Fast randomized matrix and tensor interpolative decomposition using CountSketch

Malik, Osman Asif; Becker, Stephen

doi:10.1007/s10444-020-09816-9

Cited by 8 publications

(6 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remark As mentioned in Section 1, TensorSketch can be regarded as a generalization of the well‐known CountSketch 32,33 and has been wildly used for devising the algorithms for problems with Kronecker product or Khatri‐Rao product structure, 33,34 which usually appear in the computations on tensor decompositions 26,34,35 . However, unlike the definition of TensorSketch in these works, where the big‐endian convention is adopted, in Definition 9, we employ the little‐endian convention to compute

$$ H(i)=H\left(\overline{i_1{i}_2\cdots {i}_N}\right)\kern0.60em \mathrm{and}\kern0.60em \Xi (i)=\Xi \left(\overline{i_1{i}_2\cdots {i}_N}\right).

…”

Section: Preliminariesmentioning

confidence: 99%

“…With the slices‐Hadamard product, we have the following formula, which can avoid forming the TensorSketch and implementing the matrix multiplication between large matrices when applying TensorSketch to

{\boldsymbol{G}}_{\left[2\right]}^{\ne n}

. Its proof is inspired by the works in References 32,34,35.…”

Section: Some New Findingsmentioning

confidence: 99%

See 1 more Smart Citation

Practical sketching‐based randomized tensor ring decomposition

Yu,

2024

Numerical Linear Algebra App

View full text Add to dashboard Cite

Based on sketching techniques, we propose two practical randomized algorithms for tensor ring (TR) decomposition. Specifically, on the basis of defining new tensor products and investigating their properties, the two algorithms are devised by applying the Kronecker sub‐sampled randomized Fourier transform and TensorSketch to the alternating least squares subproblems derived from the minimization problem of TR decomposition. From the former, we find an algorithmic framework based on random projection for randomized TR decomposition. We compare our proposals with the existing methods using both synthetic and real data. Numerical results show that they have quite decent performance in accuracy and computing time.

show abstract

$$ H(i)=H\left(\overline{i_1{i}_2\cdots {i}_N}\right)\kern0.60em \mathrm{and}\kern0.60em \Xi (i)=\Xi \left(\overline{i_1{i}_2\cdots {i}_N}\right).

…”

Section: Preliminariesmentioning

confidence: 99%

{\boldsymbol{G}}_{\left[2\right]}^{\ne n}

. Its proof is inspired by the works in References 32,34,35.…”

Section: Some New Findingsmentioning

confidence: 99%

Practical sketching‐based randomized tensor ring decomposition

Yu,

2024

Numerical Linear Algebra App

View full text Add to dashboard Cite

show abstract

“…The count-sketch technique is used in [109] for tensor interpolative decomposition [110] and also in [111] for the CP decomposition. The concept of Higher-order Count Sketch is developed in [39] for higher order tensors to fully exploit the multidimensional structure of the data tensor.…”

Section: Randomized Count-sketch Tucker Decompositionmentioning

confidence: 99%

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

et al. 2021

View full text Add to dashboard Cite

Big data analysis has become a crucial part of new emerging technologies such as the internet of things, cyber-physical analysis, deep learning, anomaly detection, etc. Among many other techniques, dimensionality reduction plays a key role in such analyses and facilitates feature selection and feature extraction. Randomized algorithms are efficient tools for handling big data tensors. They accelerate decomposing large-scale data tensors by reducing the computational complexity of deterministic algorithms and the communication among different levels of memory hierarchy, which is the main bottleneck in modern computing environments and architectures. In this paper, we review recent advances in randomization for computation of Tucker decomposition and Higher Order SVD (HOSVD). We discuss random projection and sampling approaches, single-pass and multi-pass randomized algorithms and how to utilize them in the computation of the Tucker decomposition and the HOSVD. Simulations on synthetic and real datasets are provided to compare the performance of some of best and most promising algorithms.

show abstract

“…Their work aimed to speed up the traditional ALS algorithm via randomized least-squares regressions. With respect to Tucker decomposition, Malik et al proposed two randomized algorithms using TensorSketch for low-rank tensor decomposition [48]. Che et al designed an effective randomized algorithm for computing the low-rank approximation of tensors under the sequentially truncated HOSVD (ST-HOSVD) model [49].…”

Section: Introductionmentioning

confidence: 99%

Tracking Online Low-Rank Approximations of Higher-Order Incomplete Streaming Tensors

Thanh¹,

abed-meraim²,

Linh-Trung³

et al. 2023

Preprint

View full text Add to dashboard Cite

Abstract: In this paper, we propose two new provable algorithms for tracking online low-rank approximations of high-order streaming tensors with missing data. The first algorithm, dubbed adaptive Tucker decomposition (ATD), minimizes a weighted recursive least-squares cost function to obtain the tensor factors and the core tensor in an efficient way, thanks to the alternating minimization framework and the randomized sketching technique. Under the Canonical Polyadic (CP) model, the second algorithm called ACP is developed as a variant of ATD when the core tensor is imposed to be identity. Both algorithms are low-complexity tensor trackers that have fast convergence and low memory storage requirements. A unified convergence analysis is presented for ATD and ACP to justify their performance. Experiments indicate that the two proposed algorithms are capable of streaming tensor decomposition with competitive performance with respect to estimation accuracy and runtime on both synthetic and real data. Code: https://github.com/thanhtbt/tensor_tracking Comment: to appear in Elsevier Patterns

show abstract

Fast randomized matrix and tensor interpolative decomposition using CountSketch

Cited by 8 publications

References 48 publications

Practical sketching‐based randomized tensor ring decomposition

Practical sketching‐based randomized tensor ring decomposition

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

Tracking Online Low-Rank Approximations of Higher-Order Incomplete Streaming Tensors

Contact Info

Product

Resources

About