Multi-Objective Loss Balancing for Physics-Informed Deep Learning

Bischof, Rafael; Kraus, Michael

doi:10.13140/rg.2.2.20057.24169

Cited by 5 publications

(10 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This technique takes advantage of the fact that neural networks can approximate the solutions of differential equations by the universal approximation theorem, as well as the fact that differential equations can be encoded into the loss function using the autodiff algorithm. Following the notation of [2], consider the following general parameterised PDE problem:…”

Section: Physics Informed Neural Network (Pinns)mentioning

confidence: 99%

“…The question of selecting λ i 's to optimize the performance of PINNs has been addressed in [2], and we will briefly summarize the state of the art techniques. First, we give the following definitions from multi-objective optimization theory.…”

Section: Loss Balance and Multi-objective Optimizationmentioning

confidence: 99%

“…The state of the art algorithm presented in [2] is called Relative Loss Balancing with Random Lookback (ReLoBRaLo). This algorithm combines two other loss balancing algorithms and introduces a random lookback.…”

Section: Adaptive Algorithmsmentioning

confidence: 99%

“…where ρ is a Bernoulli random variable with E(ρ) chosen to be close to 1, i as the first term. This algorithm is extensively tested in [2], and was shown to reach better testing accuracy than other algorithms with certain PDE problems. In particular, it outperformed other loss balance algorithms by an entire order of magnitude when tested on the Kirchhoff Plate Bending Equation.…”

Section: Adaptive Algorithmsmentioning

confidence: 99%

“…We believe that having a loss space which is dynamic in this way could help the training process traverse complex loss functions. In particular, the authors of [2] claim that the random lookback allows the model to escape from local minima in the loss space. We believe that having a stochastic loss space could help the model not only escape local minima, but also to keep the loss terms balanced.…”

Section: Stochastic Combinationsmentioning

confidence: 99%

See 4 more Smart Citations

Stochastic Scaling in Loss Functions for Physics-Informed Neural Networks

Mills¹,

Pozdnyakov²

2022

Preprint

View full text Add to dashboard Cite

Differential equations are used in a wide variety of disciplines, describing the complex behavior of the physical world. Analytic solutions to these equations are often difficult to solve for, limiting our current ability to solve complex differential equations and necessitating sophisticated numerical methods to approximate solutions. Trained neural networks act as universal function approximators, able to numerically solve differential equations in a novel way. In this work, methods and applications of neural network algorithms for numerically solving differential equations are explored, with an emphasis on varying loss functions and biological applications. Variations on traditional loss function and training parameters show promise in making neural network-aided solutions more efficient, allowing for the investigation of more complex equations governing biological principles.

show abstract

Section: Physics Informed Neural Network (Pinns)mentioning

confidence: 99%

Section: Loss Balance and Multi-objective Optimizationmentioning

confidence: 99%

Section: Adaptive Algorithmsmentioning

confidence: 99%

Section: Adaptive Algorithmsmentioning

confidence: 99%

Section: Stochastic Combinationsmentioning

confidence: 99%

See 3 more Smart Citations

Stochastic Scaling in Loss Functions for Physics-Informed Neural Networks

Mills¹,

Pozdnyakov²

2022

Preprint

View full text Add to dashboard Cite

show abstract

NN‐mCRE: A modified constitutive relation error framework for unsupervised learning of nonlinear state laws with physics‐augmented neural networks

Benady,

Baranger,

Chamoin

2024

Numerical Meth Engineering

View full text Add to dashboard Cite

This article proposes a new approach to train physics‐augmented neural networks with observable data to represent mechanical constitutive laws. To train the neural network and learn thermodynamics potentials, the proposed method does not rely on strain‐stress or strain‐free energy pairs but needs only partial strain or displacement measurements inside the structure. The neural network is trained thanks to an unsupervised procedure in which the modified constitutive relation error (mCRE) is minimized. The mCRE functional provides a bias‐aware data assimilation framework with a rich physical sense as the constitutive relation error (CRE) part can be interpreted as a modeling error continuously defined over the structure, and can be used as a prediction quality in the inference phase. This article also extends previous works on the mCRE by introducing a new minimization procedure in the case of nonlinear state laws. As typical structural health monitoring applications may require that the neural networks should be trained online, an important focus is thus made on automatic and adaptive tuning of sensitive hyperparameters (learning rate, weighting between losses, number of epochs and initialization). It is shown that when the training database is rich enough with respect to the loading cases, the proposed method achieves remarkable performance regarding the quality of the learned model, noise robustness, and low sensitivity to user‐defined hyperparameters. The method is evaluated on two test cases: a non‐quadratic potential in the small strain regime with synthetic optic fiber measurements, and a Mooney–Rivlin model in the hyperelastic case with synthetic digital image correlation observations.

show abstract

Manipulating the loss calculation to enhance the training process of physics-informed neural networks to solve the 1D wave equation

Nosrati,

Emami Niri

2023

Engineering with Computers

View full text Add to dashboard Cite

Multi-Objective Loss Balancing for Physics-Informed Deep Learning

Cited by 5 publications

References 39 publications

Stochastic Scaling in Loss Functions for Physics-Informed Neural Networks

Stochastic Scaling in Loss Functions for Physics-Informed Neural Networks

NN‐mCRE: A modified constitutive relation error framework for unsupervised learning of nonlinear state laws with physics‐augmented neural networks

Manipulating the loss calculation to enhance the training process of physics-informed neural networks to solve the 1D wave equation

Contact Info

Product

Resources

About