A multi-stage deep learning based algorithm for multiscale model reduction

Chung, Eric T.; Leung, Wing Tat; Pun, Sai-Mang; Zhang, Zecheng

doi:10.1016/j.cam.2021.113506

Cited by 15 publications

(14 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, deep learning algorithms have strong fusion and changeable algorithms. The use of a deep learning network to process the information of the prediction model is complicated, which is main shortcoming of this study [38]. Therefore, the study hereby puts forward expectations that the prediction of carbon emission information is crucial to the country's mid-to-long-term "carbon peak" strategy.…”

Section: Discussionmentioning

confidence: 97%

The Prediction of Carbon Emission Information in Yangtze River Economic Zone by Deep Learning

Huafang

Cheng

2021

Land

View full text Add to dashboard Cite

This study aimed to respond to the national “carbon peak” mid-and long-term policy plan, comprehensively promote energy conservation and emission reduction, and accurately manage and predict carbon emissions. Firstly, the proposed method analyzes the Yangtze River Economic Belt as well as its “carbon peak” and carbon emissions. Secondly, a support vector regression (SVR) machine prediction model is proposed for the carbon emission information prediction of the Yangtze River Economic Zone. This experiment uses a long short-term memory neural network (LSTM) to train the model and realize the experiment’s prediction of carbon emissions. Finally, this study obtained the fitting results of the prediction model and the training model, as well as the prediction results of the prediction model. Information indicators such as the scale of industry investment, labor efficiency output, and carbon emission intensity that affect carbon emissions in the “Yangtze River Economic Belt” basin can be used to accurately predict the carbon emissions information under this model. Therefore, the experiment shows that the SVR model for solving complex nonlinear problems can achieve a relatively excellent prediction effect under the training of LSTM. The deep learning model adopted herein realized the accurate prediction of carbon emission information in the Yangtze River Economic Zone and expanded the application space of deep learning. It provides a reference for the model in related fields of carbon emission information prediction, which has certain reference significance.

show abstract

Section: Discussionmentioning

confidence: 97%

The Prediction of Carbon Emission Information in Yangtze River Economic Zone by Deep Learning

Huafang

Cheng

2021

Land

View full text Add to dashboard Cite

show abstract

“…) are the energy function estimators of the two particles. However, using the unbiased estimators for the energy, Û (θ 1 ) and Û (θ 2 ), in re-LD with discretized dynamics (11) leads to a large bias for the estimator of the swapping rate r(θ 1 t , θ 2 2 ) defined in (8). To remove the bias from the swaps, we allow the particles swapping (θ 1 k+1 , θ 2 k+1 ) = (θ 2 k+1 , θ 1 k+1 ) with the following unbiased rate estimator [12],…”

Section: Errors In the Energy Functionmentioning

confidence: 99%

“…For example, in petroleum engineering applications, engineers seek to calculate, using the porous media equations, the pressure field of the oil based on its permeability. In practice, the value of the oil permeability varies frequently; hence, one needs to calculate the oil pressure field for a distribution of fast-varying permeabilities [29,9,11]. The traditional numerical frameworks require intense computations to solve these parametric PDEs.…”

Section: Introductionmentioning

confidence: 99%

Accelerated replica exchange stochastic gradient Langevin diffusion enhanced Bayesian DeepONet for solving noisy parametric PDEs

Lin¹,

Moya²,

Zhang³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The Deep Operator Networks (DeepONet) is a fundamentally different class of neural networks that we train to approximate nonlinear operators, including the solution operator of parametric partial differential equations (PDE). DeepONets have shown remarkable approximation and generalization capabilities even when trained with relatively small datasets. However, the performance of DeepONets deteriorates when the training data is polluted with noise, a scenario that occurs very often in practice. To enable DeepONets training with noisy data, we propose using the Bayesian framework of replica-exchange Langevin diffusion. Such a framework uses two particles, one for exploring and another for exploiting the loss function landscape of DeepONets. We show that the proposed framework's exploration and exploitation capabilities enable (1) improved training convergence for DeepONets in noisy scenarios and (2) attaching an uncertainty estimate for the predicted solutions of parametric PDEs. In addition, we show that replica-exchange Langeving Diffusion (remarkably) also improves the DeepONet's mean prediction accuracy in noisy scenarios compared with vanilla Deep-ONets trained with state-of-the-art gradient-based optimization algorithms (e.g., Adam). To reduce the potentially high computational cost of replica, in this work, we propose an accelerated training framework for replica-exchange Langevin diffusion that exploits the neural network architecture of DeepONets to reduce its computational cost up to 25% without compromising the proposed framework's performance. Finally, we illustrate the effectiveness of the proposed Bayesian framework using a series of experiments on four parametric PDE problems.

show abstract

“…One of the contributions of this paper is the use of machine learning to accelerate the computations and discuss a possible design for machine learning [4,28,2,22,21]. We again comment that the computation of u 1 is more computationally difficult compared to u 2 as it uses implicit discretization.…”

Section: Introductionmentioning

confidence: 97%

HEI: hybrid explicit-implicit learning for multiscale problems

Efendiev,

Leung,

Lin

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Splitting method is a powerful method to handle application problems by splitting physics, scales, domain, and so on. Many splitting algorithms have been designed for efficient temporal discretization. In this paper, our goal is to use temporal splitting concepts in designing machine learning algorithms and, at the same time, help splitting algorithms by incorporating data and speeding them up. Since the spitting solution usually has an explicit and implicit part, we will call our method hybrid explicit-implict (HEI) learning. We will consider a recently introduced multiscale splitting algorithms, where the multiscale problem is solved on a coarse grid. To approximate the dynamics, only a few degrees of freedom are solved implicitly, while others explicitly. This splitting concept allows identifying degrees of freedom that need implicit treatment. In this paper, we use this splitting concept in machine learning and propose several strategies. First, the implicit part of the solution can be learned as it is more difficult to solve, while the explicit part can be computed. This provides a speed-up and data incorporation for splitting approaches. Secondly, one can design a hybrid neural network architecture because handling explicit parts requires much fewer communications among neurons and can be done efficiently. Thirdly, one can solve the coarse grid component via PDEs or other approximation methods and construct simpler neural networks for the explicit part of the solutions. We discuss these options and implement one of them by interpreting it as a machine translation task. This interpretation of the splitting scheme successfully enables us using the Transformer since it can perform model reduction for multiple time series and learn the connection between them. We also find that the splitting scheme is a great platform to predict the coarse solution with insufficient information of the target model: the target problem is partially given and we need to solve it through a known problem which approximates the target. Our machine learning model can incorporate and encode the given information from two different problems and then solve the target problems. We conduct four numerical examples and the results show that our method is stable and accurate.

show abstract

A multi-stage deep learning based algorithm for multiscale model reduction

Cited by 15 publications

References 32 publications

The Prediction of Carbon Emission Information in Yangtze River Economic Zone by Deep Learning

The Prediction of Carbon Emission Information in Yangtze River Economic Zone by Deep Learning

Accelerated replica exchange stochastic gradient Langevin diffusion enhanced Bayesian DeepONet for solving noisy parametric PDEs

HEI: hybrid explicit-implicit learning for multiscale problems

Contact Info

Product

Resources

About