Personalized Algorithm Generation: A Case Study in Meta-Learning ODE Integrators

Abstract

We study the meta-learning of numerical algorithms for scientific computing, which combines the mathematically driven, handcrafted design of general algorithm structure with a data-driven adaptation to specific classes of tasks. This represents a departure from the classical approaches in numerical analysis, which typically do not feature such learning-based adaptations. As a case study, we develop a machine learning approach that automatically learns effective solvers for initial value problems in the form of ordinary differential equations (ODEs), based on the Runge-Kutta (RK) integrator architecture. By combining neural network approximations and meta-learning, we show that we can obtain highorder integrators for targeted families of differential equations without the need for computing integrator coefficients by hand. Moreover, we demonstrate that in certain cases we can obtain superior performance to classical RK methods. This can be attributed to certain properties of the ODE families being identified and exploited by the approach. Overall, this work demonstrates an effective, learning-based approach to the design of algorithms for the numerical solution of differential equations, an approach that can be readily extended to other numerical tasks.

Walking into the complex plane to "order" better time integrators

“…This may be an intensive symbolic task for more complicated problems. However, the right complex steps could be of 'learned' for specific and difficult problems through approaches similar to [17] where a machine learning approach is used to obtain Runge-Kutta methods tailored to particular ODE families.…”

Principled Acceleration of Iterative Numerical Methods Using Machine Learning

“…In [20], authors report a significant error reduction by correcting solutions using neural networks trained with PDE solvers. In this line of research, there are several works where meta-learning approaches are taken to solve computational problems [23,25,26,27,28,29]. Meta-learning, or learning to learn, leverages previous learning experiences to improve future learning performance [30], which fits the motivation of utilizing the data from previously solved equations for the next one.…”

“…Meta-learning, or learning to learn, leverages previous learning experiences to improve future learning performance [30], which fits the motivation of utilizing the data from previously solved equations for the next one. For instance, [25] uses meta-learning to generate smoothers of the Multi-grid Network for parametrized PDEs, and [26] proposes a meta-learning approach to learn effective solvers based on the Runge-Kutta method for ordinary differential equations (ODEs). In [27,28], meta-learning is used to accelerate the training of physics-informed neural networks for solving PDEs.…”