Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors

Abstract

Extracting latent low-dimensional structure from high-dimensional data is of paramount importance in timely inference tasks encountered with 'Big Data' analytics. However, increasingly noisy, heterogeneous, and incomplete datasets as well as the need for real-time processing of streaming data pose major challenges to this end. In this context, the present paper permeates benefits from rank minimization to scalable imputation of missing data, via tracking low-dimensional subspaces and unraveling latent (possibly multi-way) structure from incomplete streaming data. For low-rank matrix data, a subspace estimator is proposed based on an exponentially-weighted least-squares criterion regularized with the nuclear norm. After recasting the non-separable nuclear norm into a form amenable to online optimization, real-time algorithms with complementary strengths are developed and their convergence is established under simplifying technical assumptions. In a stationary setting, the asymptotic estimates obtained offer the well-documented performance guarantees of the batch nuclear-norm regularized estimator. Under the same unifying framework, a novel online (adaptive) algorithm is developed to obtain multi-way decompositions of low-rank tensors with missing entries, and perform imputation as a byproduct. Simulated tests with both synthetic as well as real Internet and cardiac magnetic resonance imagery (MRI) data confirm the efficacy of the proposed algorithms, and their superior performance relative to state-of-the-art alternatives.

Stochastic Subsampling for Factorizing Huge Matrices

“…Convergence of SOMF Similar to [15,34], we show that the sequence of iterates (D t ) t asymptotically reaches a critical point of the empirical risk (3). We introduce the same hypothesis on the code covariance estimationC t as in [15] and a similar one on G tthey ensure strong convexity of the surrogate and boundedness of (α t ) t .…”

“…1 Note that we solve the fully observed problem despite the use of subsampled data, unlike other recent work on low-rank factorization [34].…”

Learning from imbalanced data: open challenges and future directions

“…Additionally, interpretable classifiers that can handle massive and skewed data are of interest. -We must develop methods for processing and classifying big data in form of graphs, xml structures, video sequences, hyperspectral images, associations, tensors etc [7,37]. Such data types are becoming more and more frequent in both imbalanced and big data analytics and impose certain restrictions on machine learning systems.…”

A survey of machine learning for big data processing

“…Most recently, low-rank matrix plays a more and more central role in large-scale data analysis and dimensionality reduction [8,87]. The problem of recovering a low-rank matrix is a fundamental problem with applications in machine learning [88].…”