Stochastic Perturbation Theory

Stewart, G. W.

doi:10.1137/1032121

Cited by 333 publications

(375 citation statements)

References 30 publications

Supporting

Mentioning

369

Contrasting

Unclassified

Order By: Relevance

“…Practically, it is also unreasonable to expect it to go as tiny as the machine's unit roundoff u as the number of Lanczos steps increases. For this example, by the DavisKahan sin θ theorem [7] (see also [22]), we should expect, for 1 ď j ď 3, (observed) sin θpu j ,ũ j q " O(Lanczos approximation error)`Opu{δq, where Opu{δq is due to machine's roundoff and can dominate the Lanczos approximation error after certain number of Lanczos steps. To illustrate this point, we plot, in Figure 7.1,…”

Section: Example 72mentioning

confidence: 97%

“…A matrix norm~¨~is called a unitarily invariant norm on C mˆn if it is a matrix norm and has the following two properties [1,22] 1.…”

Section: Unitarily Invariant Normmentioning

confidence: 99%

“…Two commonly used unitarily invariant norms are the spectral norm: }B} 2 " max j σ j , the Frobenius norm: This lemma for the Frobenius norm and for the case when either eigpHq is in a closed interval that contains no eigenvalue of M or vice versa is essentially in [7] (see also [22]), and it is due to [2,3] for the most general case: eigpHq X eigpM q " H and any unitarily invariant norm.…”

Section: Unitarily Invariant Normmentioning

confidence: 99%

See 2 more Smart Citations

Convergence of the block Lanczos method for eigenvalue clusters

Zhang

2014

Numer. Math.

View full text Add to dashboard Cite

The Lanczos method is often used to solve a large scale symmetric matrix eigenvalue problem. It is well-known that the single-vector Lanczos method can only find one copy of any multiple eigenvalue and encounters slow convergence towards clustered eigenvalues. On the other hand, the block Lanczos method can compute all or some of the copies of a multiple eigenvalue and, with a suitable block size, also compute clustered eigenvalues much faster. The existing convergence theory due to Saad for the block Lanczos method, however, does not fully reflect this phenomenon since the theory was established to bound approximation errors in each individual approximate eigenpairs. Here, it is argued that in the presence of an eigenvalue cluster, the entire approximate eigenspace associated with the cluster should be considered as a whole, instead of each individual approximate eigenvectors, and likewise for approximating clusters of eigenvalues. In this paper, we obtain error bounds on approximating eigenspaces and eigenvalue clusters. Our bounds are much sharper than the existing ones and expose true rates of convergence of the block Lanczos method towards eigenvalue clusters. Furthermore, their sharpness is independent of the closeness of eigenvalues within a cluster. Numerical examples are presented to support our claims. Also a possible extension to the generalized eigenvalue problem is outlined.

show abstract

Section: Example 72mentioning

confidence: 97%

“…A matrix norm~¨~is called a unitarily invariant norm on C mˆn if it is a matrix norm and has the following two properties [1,22] 1.…”

Section: Unitarily Invariant Normmentioning

confidence: 99%

Section: Unitarily Invariant Normmentioning

confidence: 99%

See 1 more Smart Citation

Convergence of the block Lanczos method for eigenvalue clusters

Zhang

2014

Numer. Math.

View full text Add to dashboard Cite

show abstract

“…The quality of approximation bound Eq. 4 holds for any input matrix A, regardless of how A is constructed; its proof relies on matrix perturbation theory (15,16). The arbitrarily chosen failure probability can be set to any δ > 0 by repeating the algorithm O(log(1/δ)) times and taking the best of the results.…”

Section: Statistical Leverage and Improved Matrix Decompositionsmentioning

confidence: 99%

CUR matrix decompositions for improved data analysis

Mahoney

Drineas

2009

Proc. Natl. Acad. Sci. U.S.A.

618

558

View full text Add to dashboard Cite

Principal components analysis and, more generally, the Singular Value Decomposition are fundamental data analysis tools that express a data matrix in terms of a sequence of orthogonal or uncorrelated vectors of decreasing importance. Unfortunately, being linear combinations of up to all the data points, these vectors are notoriously difficult to interpret in terms of the data and processes generating the data. In this article, we develop CUR matrix decompositions for improved data analysis. CUR decompositions are low-rank matrix decompositions that are explicitly expressed in terms of a small number of actual columns and/or actual rows of the data matrix. Because they are constructed from actual data elements, CUR decompositions are interpretable by practitioners of the field from which the data are drawn (to the extent that the original data are). We present an algorithm that preferentially chooses columns and rows that exhibit high "statistical leverage" and, thus, in a very precise statistical sense, exert a disproportionately large "influence" on the best low-rank fit of the data matrix. By selecting columns and rows in this manner, we obtain improved relative-error and constant-factor approximation guarantees in worst-case analysis, as opposed to the much coarser additive-error guarantees of prior work. In addition, since the construction involves computing quantities with a natural and widely studied statistical interpretation, we can leverage ideas from diagnostic regression analysis to employ these matrix decompositions for exploratory data analysis.randomized algorithms | singular value decomposition | principal components analysis | interpretation | statistical leverage M odern datasets are often represented by large matrices since an m × n real-valued matrix A provides a natural structure for encoding information about m objects, each of which is described by n features. Examples of such objects include documents, genomes, stocks, hyperspectral images, and web groups. Examples of the corresponding features are terms, environmental conditions, temporal resolution, frequency resolution, and individual web users. In many cases, an important step in the analysis of such data is to construct a compressed representation of A that may be easier to analyze and interpret in light of a corpus of field-specific knowledge. The most common such representation is obtained by truncating the Singular Value Decomposition (SVD) at some number k min{m, n} terms. For example, Principal Components Analysis (PCA) is just this procedure applied to a suitably normalized data correlation matrix.Recall the SVD of a general matrix A ∈ R m×n . Given A,∈ R m and {v t } n t=1 ∈ R n are such that The SVD is widely used in data analysis, often via methods such as PCA, in large part because the subspaces spanned by the vectors (typically obtained after truncating the SVD to some small number k of terms) provide the best rank-k approximation to the data matrix A. If k ≤ r = rank(A) and we define A k = k t=1 σ t u t v t T , then A − A ...

show abstract

“…To bound the right-hand side, we can invoke a well-known theorem from perturbation theory of matrix inverses, Theorem III.2.5 from [28]. Assuming that C −1 E 2 < 1, the theorem, submultiplicativity of matrix norms, and the fact that C…”

Section: Matrix Perturbation Boundsmentioning

confidence: 99%

Algorithms for subset selection in linear regression

Das

Kempe

2008

Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing

162

158

View full text Add to dashboard Cite

We study the problem of selecting a subset of k random variables to observe that will yield the best linear prediction of another variable of interest, given the pairwise correlations between the observation variables and the predictor variable. Under approximation preserving reductions, this problem is also equivalent to the "sparse approximation" problem of approximating signals concisely.We propose and analyze exact and approximation algorithms for several special cases of practical interest. We give an FPTAS when the covariance matrix has constant bandwidth, and exact algorithms when the associated covariance graph, consisting of edges for pairs of variables with non-zero correlation, forms a tree or has a large (known) independent set. Furthermore, we give an exact algorithm when the variables can be embedded into a line such that the covariance decreases exponentially in the distance, and a constant-factor approximation when the variables have no "conditional suppressor variables".Much of our reasoning is based on perturbation results for the R 2 multiple correlation measure, frequently used as a measure for "goodness-of-fit statistics". It lies at the core of our FPTAS, and also allows us to extend exact algorithms to approximation algorithms when the matrix "nearly" falls into one of the above classes. We also use perturbation analysis to prove approximation guarantees for the widely used "Forward Regression" heuristic when the observation variables are nearly independent.

show abstract

Stochastic Perturbation Theory

Cited by 333 publications

References 30 publications

Convergence of the block Lanczos method for eigenvalue clusters

Convergence of the block Lanczos method for eigenvalue clusters

CUR matrix decompositions for improved data analysis

Algorithms for subset selection in linear regression

Contact Info

Product

Resources

About