Matrix decompositions using sub-Gaussian random matrices

Aizenbud, Yariv; Averbuch, Amir

doi:10.1093/imaiai/iay017

Cited by 12 publications

(11 citation statements)

References 39 publications

(81 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the real data sets applying the multi-view approach requires an eigen decomposition of large matrices. To reduce the runtime of experiments we us an approximate matrix decomposition based on sparse random projections [43].…”

Section: B Multi-view Clusteringmentioning

confidence: 99%

See 1 more Smart Citation

Multi-view diffusion maps

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this paper, a reduced dimensionality representation is learned from multiple views of the processed data. These multiple views can be obtained, for example, when the same underlying process is observed using several different modalities, or measured with different instrumentation. The goal is to effectively utilize the availability of such multiple views for various purposes such as non-linear embedding, manifold learning, spectral clustering, anomaly detection and non-linear system identification. The proposed method, which is called multi-view, exploits the intrinsic relation within each view as well as the mutual relations between views. This is achieved by defining a cross-view model in which an implied random walk process is restrained to hop between objects in the different views. This multi-view method is robust to scaling and it is insensitive to small structural changes in the data. Within this framework, new diffusion distances are defined to analyze the spectra of the implied kernels. The applicability of the multi-view approach is demonstrated for clustering, classification and manifold learning using both artificial and real data. 2 The problem of learning from two views has been studied in the field of spectral clustering. Most of these studies have been focused on classification and clustering that are based on spectral characteristics of the data while using two or more sampled views. Some approaches, which address this problem, are Bilinear Model [9], Partial Least Squares [10] and Canonical Correlation Analysis [11]. These methods are powerful for learning the relation between different views but do not provide separate insights or combined into the low dimensional geometry or structure of each view. Recently, a few kernel based methods (e.g [12]) propose a model of co-regularizing kernels in both views in a way that resembles joint diagonalization. It is done by searching for an orthogonal transformation that maximizes the diagonal terms of the kernel matrices obtained from all views. A penalty term, which incorporates the disagreement between clusters from the views, was added. Their algorithm is based on alternating maximization procedure. A mixture of Markov chains is proposed in [13] to model multiple views in order to apply spectral clustering. It deals with two cases in graph theory: directed and undirected graph where the second case is related to our work. This approach converges the undirected graph problem to a Markov chains averaging where each is constructed separately within the views. A way to incorporate a given multiple metrics for the same data using a cross diffusion process is described in [14]. They define a new diffusion distance which is useful for classification, clustering or retrieval tasks. However, the proposed process is not symmetrical thus does not allow to compute an embedding. An iterative algorithm for spectral clustering is proposed in [15]. The idea is to iteratively modify each view using the representation of the other view. The problem of two manifolds, ...

show abstract

Section: B Multi-view Clusteringmentioning

confidence: 99%

“…8, the views X and Y , which were generated by Eqs. (43) and (44), are presented. Color and shape indicate the ground truth clusters.…”

Section: B Multi-view Clusteringmentioning

confidence: 99%

Multi-view diffusion maps

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, as the support should be determined such that the least-squares matrix is invertible we get thatĨ ∝ O(d m ). Thus, we can use a randomized rank d SVD implementation such as the one detailed in [1] and reduce the complexity of this step to O(n ·Ĩ) +Õ(n · d 2 ), whereÕ neglects logarithmic factors of d. Plugging in the estimated size ofĨ, we get that the overall complexity of Step 1 amounts to O(n · d m ) Corollary 3.10. The overall complexity for the projection of a given point r onto the approximating manifold is O(n · d m + d 3m ).…”

Section: Complexity Of the Mmls Projectionmentioning

confidence: 99%

Manifold Approximation by Moving Least-Squares Projection (MMLS)

Sober

Levin

2019

Constr Approx

View full text Add to dashboard Cite

In order to avoid the curse of dimensionality, frequently encountered in Big Data analysis, there was a vast development in the field of linear and nonlinear dimension reduction techniques in recent years. These techniques (sometimes referred to as manifold learning) assume that the scattered input data is lying on a lower dimensional manifold, thus the high dimensionality problem can be overcome by learning the lower dimensionality behavior. However, in real life applications, data is often very noisy. In this work, we propose a method to approximate M a d-dimensional C m+1 smooth submanifold of R n (d << n) based upon noisy scattered data points (i.e., a data cloud). We assume that the data points are located "near" the lower dimensional manifold and suggest a non-linear moving least-squares projection on an approximating d-dimensional manifold. Under some mild assumptions, the resulting approximant is shown to be infinitely smooth and of high approximation order (i.e., O(h m+1 ), where h is the fill distance and m is the degree of the local polynomial approximation). The method presented here assumes no analytic knowledge of the approximated manifold and the approximation algorithm is linear in the large dimension n. Furthermore, the approximating manifold can serve as a framework to perform operations directly on the high dimensional data in a computationally efficient manner. This way, the preparatory step of dimension reduction, which induces distortions to the data, can be avoided altogether.

show abstract

“…Hence, it is often possible to zero-out the small values of W k i,j , allowing for cheaper sparse-matrix computations. Additionally, computing the eigen-decomposition in step 6 for large datasets (large N ) may be accomplished more efficiently using randomized methods [17,2].…”

Section: Algorithms Summary and Computational Costmentioning

confidence: 99%

The Steerable Graph Laplacian and its Application to Filtering Image Datasets

Landa¹,

Shkolnisky²

2018

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

In recent years, improvements in various image acquisition techniques gave rise to the need for adaptive processing methods, aimed particularly for large datasets corrupted by noise and deformations. In this work, we consider datasets of images sampled from a low-dimensional manifold (i.e. an imagevalued manifold), where the images can assume arbitrary planar rotations. To derive an adaptive and rotation-invariant framework for processing such datasets, we introduce a graph Laplacian (GL)-like operator over the dataset, termed steerable graph Laplacian. Essentially, the steerable GL extends the standard GL by accounting for all (infinitely-many) planar rotations of all images. As it turns out, similarly to the standard GL, a properly normalized steerable GL converges to the Laplace-Beltrami operator on the low-dimensional manifold. However, the steerable GL admits an improved convergence rate compared to the GL, where the improved convergence behaves as if the intrinsic dimension of the underlying manifold is lower by one. Moreover, it is shown that the steerable GL admits eigenfunctions of the form of Fourier modes (along the orbits of the images' rotations) multiplied by eigenvectors of certain matrices, which can be computed efficiently by the FFT. For image datasets corrupted by noise, we employ a subset of these eigenfunctions to "filter" the dataset via a Fourier-like filtering scheme, essentially using all images and their rotations simultaneously. We demonstrate our filtering framework by de-noising simulated single-particle cryo-EM image datasets.

show abstract

Matrix decompositions using sub-Gaussian random matrices

Cited by 12 publications

References 39 publications

Multi-view diffusion maps

Multi-view diffusion maps

Manifold Approximation by Moving Least-Squares Projection (MMLS)

The Steerable Graph Laplacian and its Application to Filtering Image Datasets

Contact Info

Product

Resources

About