ISLET: Fast and Optimal Low-Rank Tensor Regression via Importance Sketching

Tensor Linear Regression: Degeneracy and Solution

“…This result is consistent with our discussions in Subsection III-A. For CP parameters penalized method (15) with g specified as the ridge penalty (16), there is only one case that is identified as divergence for small λ (λ = 0.001), which confirms that CP penalization is able to overcome the problem of degeneracy as in Subsection III-C. For the noisy situations, i.e., Cases 2a and 2b, though CP parameters penalized method (15) has some rare cases to be identified as divergence, it is significantly less than the numbers of divergent cases of the other two methods. Again, we note that the empirical rule of divergence identification is not perfect, and so the very small number of identified divergences does not necessarily contradict with our theory in Section III-C. Figure 2 shows an example that the least squares and the coefficient penalized methods provide divergent iteration sequences, while the results of CP parameters penalized methods are identified as convergence.…”

“…For notational simplicity, when the tuning parameters λ and α are given, we respectively rewrite the least squares method (9), CP parameters penalized method (15) with g specified as the ridge penalty (16), and the coefficient tensor penalized method in (18) as…”

“…In the framework of Algorithm 1, the computational cost per iteration is due to a penalized linear squares induced by g(θ) and the related tensor transformation. For example, if g(θ) is defined as (16), then one only needs to perform a tensor transformation and solve a ridge regression at each block update, which leads to the overall computational complexity…”

“…Reported are based on 50 data replications for each setting. The columns LS, λ's, and α's correspond to the least squares method (9), CP parameters penalized method (15) with g specified as the ridge penalty(16), and the coefficient penalized method(18), respectively.…”

Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD)

“…Deterministic algorithms for decomposing large-scale data tensors into the Tucker format are prohibitive and require high memory and computational complexity or only applicable for structured data tensors [15], [16]. It has been proved that randomized algorithms can tackle this difficulty by exploiting only a part of the data tensors with applications in tensor regression [17], tensor completion [18], [19] and deep learning [20]. Because of this property, they scale quite well to the tensor dimensions and orders.…”