Randomized Algorithms for Low-Rank Tensor Decompositions in the Tucker Format

Abstract

Many applications in data science and scientific computing involve large-scale datasets that are expensive to store and compute with, but can be efficiently compressed and stored in an appropriate tensor format. In recent years, randomized matrix methods have been used to efficiently and accurately compute low-rank matrix decompositions. Motivated by this success, we focus on developing randomized algorithms for tensor decompositions in the Tucker representation. Specifically, we present randomized versions of two well-known compression algorithms, namely, HOSVD and STHOSVD. We present a detailed probabilistic analysis of the error of the randomized tensor algorithms. We also develop variants of these algorithms that tackle specific challenges posed by large-scale datasets. The first variant adaptively finds a low-rank representation satisfying a given tolerance and it is beneficial when the target-rank is not known in advance. The second variant preserves the structure of the original tensor, and is beneficial for large sparse tensors that are difficult to load in memory. We consider several different datasets for our numerical experiments: synthetic test tensors and realistic applications such as the compression of facial image samples in the Olivetti database and word counts in the Enron email dataset.

Low-Rank Tucker Approximation of a Tensor from Streaming Data

“…This method compresses the target tensor after extracting each factor matrix. The resulting algorithm can be accelerated using randomized matrix approximations [27] but seems to require N passes over the tensor. Hence the method is difficult to implement when the data is too large to store locally.…”

“…In the first step of the HOSVD, Algorithm 2.1, we approximately compute the top r n eigenvectors of each matricization X pnq using the randomized SVD [16]. Indeed, the same idea was proposed in concurrent work [27], which extends the idea to the ST-HOSVD and provides an error analysis. The error analyses of the two papers essentially coincide for Algorithm 4.2.…”

“…The error analyses of the two papers essentially coincide for Algorithm 4.2. One major difference is that the authors of [27] focus on the computational benefits gained using the randomized SVD, while here we focus primarily on the benefits due to reduced storage.…”

“…‚ We propose a two-pass algorithm that uses additional data access to improve on the one-pass method. This two-pass algorithm was also proposed in the simultaneous work [27]. Both the one-pass and two-pass methods are appropriate for limited memory or distributed data settings.…”

Error-Robust Distributed Denial of Service Attack Detection Based on an Average Common Feature Extraction Technique

“…Step 1 of Algorithm 1 corresponds to the HOSVD of the dataset tensor and presents computational complexity given by [ 33 ]: where, for simplicity of notation, corresponds to the number of dataset instances M .…”