Fast online low-rank tensor subspace tracking by CP decomposition using recursive least squares from incomplete observations

Abstract

We consider the problem of online subspace tracking of a partially observed high-dimensional data stream corrupted by noise, where we assume that the data lie in a low-dimensional linear subspace. This problem is cast as an online low-rank tensor completion problem. We propose a novel online tensor subspace tracking algorithm based on the CANDECOMP/PARAFAC (CP) decomposition, dubbed OnLine Low-rank Subspace tracking by TEnsor CP Decomposition (OL-STEC). The proposed algorithm especially addresses the case in which the subspace of interest is dynamically time-varying. To this end, we build up our proposed algorithm exploiting the recursive least squares (RLS), which is the second-order gradient algorithm. Numerical evaluations on synthetic datasets and real-world datasets such as communication network traffic, environmental data, and surveillance videos, show that the proposed OLSTEC algorithm outperforms state-of-the-art online algorithms in terms of the convergence rate per iteration. approach with tensor decomposition [5,6] has gained great attentions recently because of superior performance in practice irrespective of introduction of local minima. This performance also derives from the success of matrix cases [7,8,9]. This paper follows the same line as that of the latter approach.When the data are acquired sequentially from time to time, it is more challenging because of the need for online-based analysis without storing all of the past data as well as without reliance on the batch-based process. From this perspective, the batch-based SVD approach is inefficient. It cannot be applied for real-time processing. For this problem, online subspace tracking plays a fundamentally important role in various data analyses to avoid expensive repetitive computations and high memory/storage consumption.This present paper particularly addresses two special but realistic situations that arise in the online subspace tracking in practical applications. First, (i) considering the time-varying dynamic nature of real-world streaming data, because there might not exist a unique and stationary subspace over time, we are often required to update such a time-varying subspace from moment to moment without sweeping the data in multiple times. Despite allowing moderate accuracy of subspace estimation, this update makes existing batch-based algorithms useless. In fact, as experiments described later in the paper reveal, such a batch-based approach does not work well under the situation where a stationary subspace does not exist. Furthermore, (ii) considering the situation and applications where the computational speed is much faster than the data acquiring speed, we prefer the algorithm with faster convergence rate in terms of iteration rather than that with faster computational speed. For all of these reasons, we particularly address the recursive least squares (RLS) algorithm. Although the RLS does not give higher precision from the viewpoint of the optimization theory [10], it fits the dynamic situation as considered herein because i...

A Fast Randomized Adaptive CP Decomposition For Streaming Tensors

“…In this section, we demonstrate the effectiveness and efficiency of our algorithm, ROLCP, on both synthetic and real data. We also compare ROLCP with the state-of-theart adaptive (online) CP algorithms, including PARAFAC-SDT [5], PARAFAC-RLST [5], OLCP [9], SOAP [7] and OLSTEC [11]. Default parameters of these algorithms are kept to have a fair comparison.…”

“…Smith et al have introduced an adaptive algorithm for handling streaming sparse tensors called CP-stream [10]. Kasai has recently developed an efficient second-order algorithm to exploit the recursive least squares algorithm, called OL-STEC in [11]. Among these algorithms, OLSTEC provides a competitive performance in terms of estimation accuracy.…”

Multi-source and Multi-fault Condition Monitoring Based on Parallel Factor Analysis and Sequential Probability Ratio Test

“…x to represent any element in the three-dimensional matrix. At present, there is no definite name naming subscript k, let's call it "page" [39] . The subscript of the three-dimensional matrix consists of three index value row, column, page composition.…”

Robust Object Tracking via Reverse Low-Rank Sparse Learning and Fractional-Order Variation Regularization

“…Low-rank constraint [1][2][3][4] on the candidate particles can reflect the subspace structure feature of the object appearance. is subspace representation is robust to handle the global appearance changes problem (e.g., illumination variations and pose changes).…”