Data-driven kernel designs for optimized greedy schemes: A machine learning perspective

Abstract

Thanks to their easy implementation via Radial Basis Functions (RBFs), meshfree kernel methods have been proved to be an effective tool for e.g. scattered data interpolation, PDE collocation, classification and regression tasks. Their accuracy might depend on a length scale hyperparameter, which is often tuned via cross validation schemes. Here we leverage approaches and tools from the machine learning community to introduce two-layered kernel machines, which generalize the classical RBF approaches that rely on a single hyperparameter. Indeed, the proposed learning strategy returns a kernel that is optimized not only in the Euclidean directions, but that further incorporates kernel rotations. The kernel optimization is shown to be robust by using recently improved calculations of cross validation scores. Finally, the use of greedy approaches, and specifically of the Vectorial Kernel Orthogonal Greedy Algorithm (VKOGA), allows us to construct an optimized basis that adapts to the data. Beyond a rigorous analysis on the convergence of the so-constructed two-Layered (2L)-VKOGA, its benefits are highlighted on both synthesized and real benchmark data sets.

Goal‐Oriented Two‐Layered Kernel Models as Automated Surrogates for Surface Kinetics in Reactor Simulations

“…In order to adapt the kernel to the considered dataset (X N , Y N ), recently two-layered kernels were introduced [7,8], which can be understood as a generalization of standard shape parameter-tuned kernels. Indeed, such a twolayered kernel is given by…”

“…We show that, in contrast to standard machine learning methods, the VKOGA reduces the demand for costly training data and provides accurate predictions, even outside the training range. To do so, we apply the recently developed two-layer kernels [7,8] and extend the VKOGA procedure by goal-oriented selection criteria.…”

Application of Deep Kernel Models for Certified and Adaptive RB-ML-ROM Surrogate Modeling

“…In order to make the best out of both approaches, the idea is to combine deep learning methods such as neural networks with the benefits of "shallow" kernel methods. In the following Section 4, we review two recently introduced algorithms in this direction: SDKNs [10] as well as 2L-VKOGA [9].…”

“…As analyzed in [10], these SDKNs enjoy universal approximation properties in various limit cases. The 2L-VKOGA [9] combines the use of an optimized two-layered kernel with greedy kernel algorithms, where we leverage the VKOGA algorithm [7]: Radial basis function kernels are a popular class of kernels which are given as k(x, z) = Φ( x − z ) for some radial basis function Φ : R d → R, e.g. the Gaussian kernel k(x, z) = exp(− x − z 2 ).…”

“…( 3), the VKOGA algorithm selects a suitable subset X n ⊂ X N of the dataset, such that the resulting kernel model consists only of n ≪ N summands and is thus sparse. For our current application, we extend the 2L-VKOGA approach that was only introduced for scalar valued target data [9] to vectorial data, i.e. Y N ⊂ R b .…”