Convergence of Physics-Informed Neural Networks Applied to Linear Second-Order Elliptic Interface Problems

Wu, Sidi; Zhu, Anning; null, Yifa Tang; Lu, Benzhuo

doi:10.4208/cicp.oa-2022-0218

Cited by 12 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to assess the effectiveness of the proposed method, we have examined several benchmark partial differential equations, which include the Allen-Cahn equation, the Burgers equation, a convection dominated diffusion equation, the Helmholtz equation, and some multiple-dimensional elliptic equations. We compare our method with the vanilla PINNs [31] and the Residual-based Adaptive Refinement (RAR) method [47], which conducts denser sampling in regions with large residual losses. Specifically, the settings of RAR in this paper follow the settings in the official DeepXDE tutorial [26].…”

Section: Methodsmentioning

confidence: 99%

“…One particularly noteworthy development is the emergence of the Physical-Informed Neural Network (PINN) paradigm [3,6,8,27,31,39,46,47,52]. The core concept underpinning PINNs revolves around the integration of partial differential equations and their associated conditions as soft constraints within the loss function of a neural network [21,22,25,38].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficiently Training Physics-Informed Neural Networks via Anomaly-Aware Optimization

Li,

null,

Chen

2024

NMTMA

View full text Add to dashboard Cite

Physics-Informed Neural Networks (PINNs) encounter challenges in dealing with imbalanced training losses, especially when there are sample points with extremely high losses. This can make the optimization process unstable, making it challenging to find the correct descent direction during training. In this paper, we propose a progressive learning approach based on anomaly points awareness to improve the optimization process of PINNs. Our approach comprises two primary steps: the awareness of anomaly data points and the update of training set. Anomaly points are identified by utilizing an upper bound calculated from the mean and standard deviation of the feedforward losses of all training data. In the absence of anomalies, the parameters of the PINN are optimized using the default training data; however, once anomalies are detected, a progressive exclusion method aligned with the network learning pattern is introduced to exclude potentially unfavorable data points from the training set. In addition, intermittent detection is employed, rather than performing anomaly detection in each iteration, to balance performance and efficiency. Extensive experimental results demonstrate that the proposed method leads to substantial improvement in approximation accuracy when solving typical benchmark partial differential equations. The code is accessible at https://github.com/JcLimath/Anomaly-Aware-PINN.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Efficiently Training Physics-Informed Neural Networks via Anomaly-Aware Optimization

Li,

null,

Chen

2024

NMTMA

View full text Add to dashboard Cite

show abstract

“…In recent years, there has been a surge of interest in applying neural networks in traditional scientific modeling (e.g. partial differential equations), which yields the so-called physics-informed neural networks [5,10,11,14,17,18,21,23,30,34,35]. The main idea of PINNs is to include physical domain knowledge as soft constraints in the empirical loss function and then use existing machine learning methodologies such as stochastic optimization, to train the model.…”

Section: Introductionmentioning

confidence: 99%

Less Emphasis on Hard Regions: Curriculum Learning of PINNs for Singularly Perturbed Convection-Diffusion-Reaction Problems

Yufeng Wang,

Cong Xu,

Min Yang

et al. 2024

EAJAM

View full text Add to dashboard Cite

Although physics-informed neural networks (PINNs) have been successfully applied in a wide variety of science and engineering fields, they can fail to accurately predict the underlying solution in slightly challenging convection-diffusion-reaction problems. In this paper, we investigate the reason of this failure from a domain distribution perspective, and identify that learning multi-scale fields simultaneously makes the network unable to advance its training and easily get stuck in poor local minima. We show that the widespread experience of sampling more collocation points in high-loss regions hardly help optimize and may even worsen the results. These findings motivate the development of a novel curriculum learning method that encourages neural networks to prioritize learning on easier non-layer regions while downplaying learning on harder regions. The proposed method helps PINNs automatically adjust the learning emphasis and thereby facilitates the optimization procedure. Numerical results on typical benchmark equations show that the proposed curriculum learning approach mitigates the failure modes of PINNs and can produce accurate results for very sharp boundary and interior layers. Our work reveals that for equations whose solutions have large scale differences, paying less attention to high-loss regions can be an effective strategy for learning them accurately.

show abstract

“…Крім того, бібліотека має зручний інтерфейс, що спрощує визначення моделей, навчання та отримання результатів, що робить її доступною для використання дослідниками і інженерами з різним рівнем досвіду у глибокому навчанні. Використання бібліотеки DeepXDE на даний час досить поширене в наукових публікаціях [3,4]. Однак, певним недоліком бібліотеки є те, що ефективне її використання потребує знання мов програмування та дотримання певного внутрішнього формату запису задачі, приклад використання бібліотеки можна побачити в офіційній документації [5].…”

unclassified

Проєктування Бібліотеки Нейромережевих Методів Розв’язання Крайових Задач

Ярош,

Кудін

2024

Розвиток Наук В Умовах Нової Реальності: Проблеми Та Перспективи

View full text Add to dashboard Cite

Convergence of Physics-Informed Neural Networks Applied to Linear Second-Order Elliptic Interface Problems

Cited by 12 publications

References 0 publications

Efficiently Training Physics-Informed Neural Networks via Anomaly-Aware Optimization

Efficiently Training Physics-Informed Neural Networks via Anomaly-Aware Optimization

Less Emphasis on Hard Regions: Curriculum Learning of PINNs for Singularly Perturbed Convection-Diffusion-Reaction Problems

Проєктування Бібліотеки Нейромережевих Методів Розв’язання Крайових Задач

Contact Info

Product

Resources

About