Randomized Nyström Preconditioning

Frangella, Zachary; Tropp, Joel A.; Udell, Madeleine

doi:10.1137/21m1466244

Cited by 5 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A low-rank approximation of the kernel matrix can also be constructed as a projection onto a subset of its columns. This basic idea is underlying the Nyström method, which is traditionally used to approximate large scale kernel machines ,,− and recently also as a preconditioner. ,− Given k ≪ n inducing columns , the Nyström approximation represents the full kernel matrix according to In this expression, the matrix dimensionality is implicitly encoded in the subscript, i.e., . K kk can be symmetrically factorized to obtain the form in eq (see refs and for implementation details).…”

Section: Nystöm-type Preconditionermentioning

confidence: 99%

Reconstructing Kernel-Based Machine Learning Force Fields with Superlinear Convergence

Blücher

Mueller

Chmiela

2023

J. Chem. Theory Comput.

View full text Add to dashboard Cite

Kernel machines have sustained continuous progress in the field of quantum chemistry. In particular, they have proven to be successful in the low-data regime of force field reconstruction. This is because many equivariances and invariances due to physical symmetries can be incorporated into the kernel function to compensate for much larger data sets. So far, the scalability of kernel machines has however been hindered by its quadratic memory and cubical runtime complexity in the number of training points. While it is known that iterative Krylov subspace solvers can overcome these burdens, their convergence crucially relies on effective preconditioners, which are elusive in practice. Effective preconditioners need to partially presolve the learning problem in a computationally cheap and numerically robust manner. Here, we consider the broad class of Nystrom-type methods to construct preconditioners based on successively more sophisticated low-rank approximations of the original kernel matrix, each of which provides a different set of computational trade-offs. All considered methods aim to identify a representative subset of inducing (kernel) columns to approximate the dominant kernel spectrum.

show abstract