Theory-guided hard constraint projection (HCP): A knowledge-based data-driven scientific machine learning method

Chen, Yuntian; Huang, Duohui; Zhang, Dongxiao; Zeng, Jiang; Wang, Nanzhe; Zhang, Haoran; Yan, Jinyue

doi:10.1016/j.jcp.2021.110624

Cited by 68 publications

(16 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this case, we test whether transfer learning can achieve more accurate predictions in less time. The base task adopted to generate the pre-trained model is the common groundwater flow problem, which has been well researched and discussed in many references [13], [48], [49]. Compared with single-phase flow in oil reservoirs, it is much simpler without strong nonlinearity.…”

Section: ) Test Of Single-phase Reservoir Flow With Curriculum Learningmentioning

confidence: 99%

AutoKE: An Automatic Knowledge Embedding Framework for Scientific Machine Learning

Chen

Zhang

2023

IEEE Trans. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

Imposing physical constraints on neural networks as a method of knowledge embedding has achieved great progress in solving physical problems described by governing equations. However, for many engineering problems, governing equations often have complex forms, including complex partial derivatives or stochastic physical fields, which results in significant inconveniences from the perspective of implementation. In this paper, to effectively automate the process of embedding physical knowledge, a scientific machine learning framework, called AutoKE, is proposed, and a reservoir flow problem with a complex governing equation is taken as an instance. In AutoKE, an emulator comprised of deep neural networks (DNNs) is built for predicting the physical variables of interest. An arbitrarily complex equation can be parsed and automatically converted into a computational graph through the equation parser module, and the fitness of the emulator to the governing equation is evaluated via automatic differentiation. Furthermore, the fixed weights in the loss function are substituted with adaptive weights by incorporating the Lagrangian dual method. Neural architecture search (NAS) is also introduced into the AutoKE to select an optimal network architecture of the emulator according to the specific problem. Finally, we apply transfer learning to enhance the scalability of the emulator. In experiments, the framework is verified by a series of physical problems in which it can automatically embed physical knowledge into an emulator without heavy hand-coding. The results demonstrate that the emulator can not only make accurate predictions, but also be applied to similar problems with high efficiency via transfer learning.Impact Statement -Embedding physical knowledge into machine learning has been widely applied in solving scientific computing problems. However, it is tedious and timeconsuming to establish the emulator and embed physical knowledge into it. To this end, we proposed a plug-and-play framework for solving complicated physical problems. The user inputs equations with arbitrary form according to the specified rules, which can be automatically parsed and transformed into a structural representation that the program can recognize. The embedding of physical knowledge, the establishment of emulators, and the adjustment of parameters are automated, which provides great convenience while obtaining accurate solutions. Moreover, the strong scalability enables the trained emulator to be applied to similar problems,

show abstract

Section: ) Test Of Single-phase Reservoir Flow With Curriculum Learningmentioning

confidence: 99%

AutoKE: An Automatic Knowledge Embedding Framework for Scientific Machine Learning

Chen

Zhang

2023

IEEE Trans. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Raissi et al proposed utilizing nonlinear PDEs in the loss function as a soft constrain (Raissi et al, 2019). Through a hard constraint projection, Chen et al proposed a framework to ensure model's predictions strictly conform to physical mechanisms (Chen et al, 2021). Based on the universal approximation theorem of operators, in Lu et al (2021), authors proposed DeepONet to learn continuous operators or complex systems.…”

Section: Physics-informed Machine Learningmentioning

confidence: 99%

An Intriguing Property of Geophysics Inversion

Yong¹,

Chen²,

Feng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Inversion techniques are widely used to reconstruct subsurface physical properties (e.g., velocity, conductivity, and others) from surface-based geophysical measurements (e.g., seismic, electric/magnetic (EM) data). The problems are governed by partial differential equations (PDEs) like the wave or Maxwell's equations. Solving geophysical inversion problems is challenging due to the ill-posedness and high computational cost. To alleviate those issues, recent studies leverage deep neural networks to learn the inversion mappings from geophysical measurements to the geophysical property directly.

show abstract

“…The first, designated as Theory-Guided Constrained Optimization, consists in integrating the theory into the cost function (Tartakovsky et al 2020, Wang et al 2020a). The second, called Theory-Guided Refinement of Outputs, consists in postprocessing the outputs of the model to make theme conform to the theory as accurately as possible (Chen et al 2020b, Hautier et al 2010, Khandelwal et al 2015. The third, designated as Theory-Guided Architecture, consists in using what is known theoretically about the phenomenon under study to design the architecture of the ML model (Daw et al 2020, Udrescu and Tegmark 2020 .…”

Section: Theory-guided ML Vs Ml Limiting Factors In Hydrogeologymentioning

confidence: 99%

“…To compute the residual ℛ of the governing equation and the residual ℛ of the Neumann boundary condition, Wang et al (2020a) applied the chain rule through automatic differentiation, but the residuals can also be approximated by discretization methods such as finite element method or finite difference method as in Chen et al (2020b). Finally, the cost function according to the TGCO method can be defined as follows:…”

Section: Theory-guided Constrained Optimisationmentioning

confidence: 99%

Review: Theory-guided machine learning applied to hydrogeology—state of the art, opportunities and future challenges

2021

View full text Add to dashboard Cite

Thanks to recent technological advances, hydrogeologists now have access to large amounts of data acquired in real time. Processing these data using traditional modelling tools is difficult and poses a number of challenges especially for tasks such as extracting useful features, uncertainty quantification or identifying links between various variables. Artificial Intelligence, and more specifically its subset Machine Learning, may represent a way of the future in hydrogeological research and applications. Unfortunately, several aspects of machine learning methods hamper its adoption as a complementary tool for hydrogeologists, namely the black-box nature of most models, an often-limited generalization ability, a hypothetical convergence, and uncertain transferability.Recently, an entirely novel paradigm in the field of machine learning has been identified: theory-guided machine learning, in which the models integrate some specific theoretical knowledge, laws or principles of the field of study. This review article set out to examine three theory-guided methods in their ability to overcome the limitations of machine learning for hydrogeological research and applications. These methods are, respectively, Theory-Guided Constrained Optimization (TGCO), Theory-Guided Refinement of Outputs (TGRO) and Theory-Guided Architecture (TGA). The analyses led to the following conclusions: the opacity of ML models can be reduced by any of the three theory-guided ML methods; convergence and generalizability can be enhanced by TGCO, TGA, or a combination of at least two of the theory-guided ML methods; no study conducted to date has made it possible to deduce the effectiveness of these methods on the transferability of ML models.

show abstract

Theory-guided hard constraint projection (HCP): A knowledge-based data-driven scientific machine learning method

Cited by 68 publications

References 34 publications

AutoKE: An Automatic Knowledge Embedding Framework for Scientific Machine Learning

AutoKE: An Automatic Knowledge Embedding Framework for Scientific Machine Learning

An Intriguing Property of Geophysics Inversion

Review: Theory-guided machine learning applied to hydrogeology—state of the art, opportunities and future challenges

Contact Info

Product

Resources

About