Meta-learning PINN loss functions

Psaros, Apostolos F.; Kawaguchi, Kenji; Karniadakis, George Em

doi:10.1016/j.jcp.2022.111121

Cited by 58 publications

(25 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An automated way to find a suitable setting is needed. To this end automated approaches such as AutoML 53 or Meta learning 54 , could be applied in the future. Moreover, theoretical guarantees are required, especially in sensitive human-health related applications.…”

Section: Discussionmentioning

confidence: 99%

Investigating molecular transport in the human brain from MRI with physics-informed neural networks

Zapf

Haubner

Kuchta

et al. 2022

Sci Rep

View full text Add to dashboard Cite

In recent years, a plethora of methods combining neural networks and partial differential equations have been developed. A widely known example are physics-informed neural networks, which solve problems involving partial differential equations by training a neural network. We apply physics-informed neural networks and the finite element method to estimate the diffusion coefficient governing the long term spread of molecules in the human brain from magnetic resonance images. Synthetic testcases are created to demonstrate that the standard formulation of the physics-informed neural network faces challenges with noisy measurements in our application. Our numerical results demonstrate that the residual of the partial differential equation after training needs to be small for accurate parameter recovery. To achieve this, we tune the weights and the norms used in the loss function and use residual based adaptive refinement of training points. We find that the diffusion coefficient estimated from magnetic resonance images with physics-informed neural networks becomes consistent with results from a finite element based approach when the residuum after training becomes small. The observations presented here are an important first step towards solving inverse problems on cohorts of patients in a semi-automated fashion with physics-informed neural networks.

show abstract

Section: Discussionmentioning

confidence: 99%

Investigating molecular transport in the human brain from MRI with physics-informed neural networks

Zapf

Haubner

Kuchta

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…We take [27] as another example. In this work, meta-learning is used to learn a loss function of the physics-informed neural network (PINN) [33] for solving PDEs.…”

Section: Organizing Related Workmentioning

confidence: 99%

“…). In [27], meta-learning is used for learning a loss function of the physics-informed neural network, shortly PINN [33]. The target equations are the following:…”

Section: Example 4 ([27]mentioning

confidence: 99%

“…In [20], authors report a significant error reduction by correcting solutions using neural networks trained with PDE solvers. In this line of research, there are several works where meta-learning approaches are taken to solve computational problems [23,25,26,27,28,29]. Meta-learning, or learning to learn, leverages previous learning experiences to improve future learning performance [30], which fits the motivation of utilizing the data from previously solved equations for the next one.…”

Section: Introductionmentioning

confidence: 99%

“…For instance, [25] uses meta-learning to generate smoothers of the Multi-grid Network for parametrized PDEs, and [26] proposes a meta-learning approach to learn effective solvers based on the Runge-Kutta method for ordinary differential equations (ODEs). In [27,28], meta-learning is used to accelerate the training of physics-informed neural networks for solving PDEs.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Principled Acceleration of Iterative Numerical Methods Using Machine Learning

Arisaka¹,

Li²

2022

Preprint

View full text Add to dashboard Cite

In science and engineering applications, it is often required to solve similar computational problems repeatedly. In such cases, we can utilize the data from previously solved problem instances to improve the efficiency of finding subsequent solutions. This offers a unique opportunity to combine machine learning (in particular, meta-learning) and scientific computing. To date, a variety of such domain-specific methods have been proposed in the literature, but a generic approach for designing these methods remains under-explored. In this paper, we tackle this issue by formulating a general framework to describe these problems, and propose a gradient-based algorithm to solve them in a unified way. As an illustration of this approach, we study the adaptive generation of parameters for iterative solvers to accelerate the solution of differential equations. We demonstrate the performance and versatility of our method through theoretical analysis and numerical experiments, including applications to incompressible flow simulations and an inverse problem of parameter estimation.

show abstract

Event‐triggered model reference adaptive control system design for SISO plants using meta‐learning‐based physics‐informed neural networks without labeled data and transfer learning

Duanyai,

Song,

Konghuayrob

et al. 2024

Adaptive Control & Signal

View full text Add to dashboard Cite

SummaryThis paper examines the controllability of a novel Lyapunov‐based model reference adaptive control (MRAC) system designed with a meta‐learning‐based physics‐informed neural network (MLPINN) for linear and nonlinear single‐input and single‐output (SISO) plants without labeled data (MLPINN‐MRAC system). It is devised with the benefits of several techniques: the integration of the identification process in a training mode into an online control mode (straightforward design); no labeled data generation for online identification by a physics‐informed neural network; the prevention of degradation in tracking performance by meta‐learning, as the system triggers a meta‐learning process only when an error threshold detects the deterioration (high efficiency and the reduction of computation cost); quick adaptation to new inputs and an updated control input for each sub‐time span by transfer learning. It is worth noting that the frequency of the meta‐learning event detection significantly affects the step response stability of the nonlinear plant. To achieve a better quality of the stabilization, more frequent event detection is necessary for both beginning and end intervals in control. Sixteen triggering events are enough to shape the acceptable step response of the nonlinear plant, and 44 triggering events achieve the plant's desired step response with its minor modeling error. While, as for the linear plant, a single triggering event is sufficient to attain its tolerable step response and modeling error, implying that intensive event detection is not critical in the identification. It is obvious that the MLPINN‐MRAC system functions well and is more beneficial and efficient for the nonlinear plant.

show abstract

Meta-learning PINN loss functions

Cited by 58 publications

References 16 publications

Investigating molecular transport in the human brain from MRI with physics-informed neural networks

Investigating molecular transport in the human brain from MRI with physics-informed neural networks

Principled Acceleration of Iterative Numerical Methods Using Machine Learning

Event‐triggered model reference adaptive control system design for SISO plants using meta‐learning‐based physics‐informed neural networks without labeled data and transfer learning

Contact Info

Product

Resources

About