Randomized algorithms for the low-rank approximation of matrices

Liberty, Edo; Woolfe, Franco; Martinsson, Per-Gunnar; Rokhlin, Vladimir; Tygert, Mark

doi:10.1073/pnas.0709640104

Cited by 463 publications

(478 citation statements)

References 579 publications

(903 reference statements)

Supporting

Mentioning

465

Contrasting

Unclassified

Order By: Relevance

“…The speed of the quantum algorithm is maximized when the training data kernel matrix is dominated by a relatively small number of principal components. We note that there exist several heuristic sampling algorithms for the SVM [29] and, more generally, for finding eigenvalues or vectors of low-rank matrices [30,31]. Information-theoretic arguments show that classically finding a low-rank matrix approximation is lower bounded by ΩðMÞ in the absence of prior knowledge [32], suggesting a similar lower bound for the least-squares SVM.…”

mentioning

confidence: 97%

Quantum Support Vector Machine for Big Data Classification

2014

View full text Add to dashboard Cite

Supervised machine learning is the classification of new data based on already classified training examples. In this work, we show that the support vector machine, an optimized binary classifier, can be implemented on a quantum computer, with complexity logarithmic in the size of the vectors and the number of training examples. In cases where classical sampling algorithms require polynomial time, an exponential speedup is obtained. At the core of this quantum big data algorithm is a nonsparse matrix exponentiation technique for efficiently performing a matrix inversion of the training data inner-product (kernel) matrix. Machine learning algorithms can be categorized along a spectrum of supervised and unsupervised learning [1][2][3][4]. In strictly unsupervised learning, the task is to find structure such as clusters in unlabeled data. Supervised learning involves a training set of already classified data, from which inferences are made to classify new data. In both cases, recent "big data" applications exhibit a growing number of features and input data. A Support Vector Machine (SVM) is a supervised machine learning algorithm that classifies vectors in a feature space into one of two sets, given training data from the sets [5]. It operates by constructing the optimal hyperplane dividing the two sets, either in the original feature space or in a higherdimensional kernel space. The SVM can be formulated as a quadratic programming problem [6], which can be solved in time proportional to O( logðϵ −1 ÞpolyðN; MÞ), with N the dimension of the feature space, M the number of training vectors, and ϵ the accuracy. In a quantum setting, binary classification was discussed in terms of Grover's search in [7] and using the adiabatic algorithm in [8][9][10][11]. Quantum learning was also discussed in [12,13].In this Letter, we show that a quantum support vector machine can be implemented with Oðlog NMÞ run time in both training and classification stages. The performance in N arises due to a fast quantum evaluation of inner products, discussed in a general machine learning context by us in [14]. For the performance in M, we reexpress the SVM as an approximate least-squares problem [15] that allows for a quantum solution with the matrix inversion algorithm [16,17]. We employ a technique for the exponentiation of nonsparse matrices recently developed in [18]. This allows us to reveal efficiently in quantum form the largest eigenvalues and corresponding eigenvectors of the training data overlap (kernel) and covariance matrices. We thus efficiently perform a low-rank approximation of these matrices [Principal Component Analysis (PCA)]. PCA is a common task arising here and in other machine learning algorithms [19][20][21]. The error dependence in the training stage is O(polyðϵ −1 K ; ϵ −1 Þ), where ϵ K is the smallest eigenvalue considered and ϵ is the accuracy. In cases when a low-rank approximation is appropriate, our quantum SVM operates on the full training set in logarithmic run time.Support vector machine.-The task for the S...

show abstract

mentioning

confidence: 97%

Quantum Support Vector Machine for Big Data Classification

2014

View full text Add to dashboard Cite

show abstract

“…For a large enough sampling, the effect of H is essentially captured with an emphasis on the correlations, therefore taking more from the signal than from the noise. This random sampling considerably reduces the size of the problem and has already been used in the analysis of very large datasets (23,26,27).…”

Section: Methodsmentioning

confidence: 99%

“…We propose here a simplified algorithm based on a random sampling of the transfer function associated with the matrix viewed as an operator. For large enough samplings, the data consistency is ensured through a variant of the Johnson-Lindenstrauss lemma (21,23).…”

Section: Methodsmentioning

confidence: 99%

Efficient denoising algorithms for large experimental datasets and their applications in Fourier transform ion cyclotron resonance mass spectrometry

Chiron¹,

Agthoven

Kieffer

et al. 2014

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Modern scientific research produces datasets of increasing size and complexity that require dedicated numerical methods to be processed. In many cases, the analysis of spectroscopic data involves the denoising of raw data before any further processing. Current efficient denoising algorithms require the singular value decomposition of a matrix with a size that scales up as the square of the data length, preventing their use on very large datasets. Taking advantage of recent progress on random projection and probabilistic algorithms, we developed a simple and efficient method for the denoising of very large datasets. Based on the QR decomposition of a matrix randomly sampled from the data, this approach allows a gain of nearly three orders of magnitude in processing time compared with classical singular value decomposition denoising. This procedure, called urQRd (uncoiled random QR denoising), strongly reduces the computer memory footprint and allows the denoising algorithm to be applied to virtually unlimited data size. The efficiency of these numerical tools is demonstrated on experimental data from high-resolution broadband Fourier transform ion cyclotron resonance mass spectrometry, which has applications in proteomics and metabolomics. We show that robust denoising is achieved in 2D spectra whose interpretation is severely impaired by scintillation noise. These denoising procedures can be adapted to many other data analysis domains where the size and/or the processing time are crucial. big data | noise reduction | matrix rank reduction | FT-ICR MS

show abstract

“…For a numerically low rank matrix, its approximate singular value decomposition can be efficiently evaluated using randomized algorithms (see e.g. [33,34,35]). The idea is briefly reviewed as below, though presented in a slightly non-standard way.…”

Section: Spectrum Sweeping Methods For Estimating Spectral Densitiesmentioning

confidence: 99%

“…(33) states that the trace of the matrix function f (A) can be accurately computed from the integral of f (s)φ σ (s), which is a now smooth function. Since the spectrum of A is assumed to be in the interval (−1, 1), the integration range in Eq.…”

Section: Application To Trace Estimation Of General Matrix Functionsmentioning

confidence: 99%

Randomized estimation of spectral densities of large matrices made accurate

Lin

2016

Numer. Math.

View full text Add to dashboard Cite

Abstract. For a large Hermitian matrix A ∈ C N ×N , it is often the case that the only affordable operation is matrix-vector multiplication. In such case, randomized method is a powerful way to estimate the spectral density (or density of states) of A. However, randomized methods developed so far for estimating spectral densities only extract information from different random vectors independently, and the accuracy is therefore inherently limitedwhere Nv is the number of random vectors.In this paper we demonstrate that the "O(1/ √ Nv) barrier" can be overcome by taking advantage of the correlated information of random vectors when properly filtered by polynomials of A. Our method uses the fact that the estimation of the spectral density essentially requires the computation of the trace of a series of matrix functions that are numerically low rank. By repeatedly applying A to the same set of random vectors and taking different linear combination of the results, we can sweep through the entire spectrum of A by building such low rank decomposition at different parts of the spectrum. Under some assumptions, we demonstrate that a robust and efficient implementation of such spectrum sweeping method can compute the spectral density accurately with O(N 2 ) computational cost and O(N ) memory cost. Numerical results indicate that the new method can significantly outperform existing randomized methods in terms of accuracy. As an application, we demonstrate a way to accurately compute a trace of a smooth matrix function, by carefully balancing the smoothness of the integrand and the regularized density of states using a deconvolution procedure.

show abstract

Randomized algorithms for the low-rank approximation of matrices

Cited by 463 publications

References 579 publications

Quantum Support Vector Machine for Big Data Classification

Quantum Support Vector Machine for Big Data Classification

Efficient denoising algorithms for large experimental datasets and their applications in Fourier transform ion cyclotron resonance mass spectrometry

Randomized estimation of spectral densities of large matrices made accurate

Contact Info

Product

Resources

About