Non-intrusive surrogate modeling for parametrized time-dependent partial differential equations using convolutional autoencoders

Nikolopoulos, Stefanos; Kalogeris, Ioannis; Papadopoulos, Vissarion

doi:10.1016/j.engappai.2021.104652

Cited by 45 publications

(29 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is essential to span the problem's parametric space effectively, thus sophisticated sampling methods are often utilized, such as the Latin Hypercube [69]. Many surrogate modeling techniques have been introduced over the past years, including linear [47,43,44] and nonlinear [37,38,50] dimensionality reduction methods. The selection of the appropriate method is problem dependent.…”

Section: Surrogate Modelmentioning

confidence: 99%

“…For instance, deep feedforward neural networks (FFNNs) have been successfully employed to construct response surfaces of quantities of interest in complex problems [32,33,34,35,36]. Convolutional neural networks (CNNs) in conjuction with FFNNs have been employed to predict the high-dimensional system response at different parameter instances [37,38,39]. In addition, recurrent neural networks demonstrated great potential in transient problems for propagating the state of the system forward in time without the need of solving systems of equations [40,41].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

AI-enhanced iterative solvers for accelerating the solution of large scale parametrized linear systems of equations

Nikolopoulos

Kalogeris

Papadopoulos

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recent advances in the field of machine learning open a new era in high performance computing. Applications of machine learning algorithms for the development of accurate and cost-efficient surrogates of complex problems have already attracted major attention from scientists. Despite their powerful approximation capabilities, however, surrogates cannot produce the 'exact' solution to the problem. To address this issue, this paper implements up-to-date ML tools and delivers customized iterative solvers of linear equation systems, capable of solving large-scale parametrized problems at any desired level of accuracy. Specifically, the proposed approach consists of the following two steps. At first, a reduced set of model evaluations is performed and the corresponding solutions are used to establish an approximate mapping from the problem's parametric space to its solution space using deep feedforward neural networks and convolutional autoencoders. This mapping serves a means to obtain very accurate initial predictions of the system's response to new query points at negligible computational cost. Subsequently, an iterative solver inspired by the Algebraic Multigrid method in combination with Proper Orthogonal Decomposition, termed POD-2G, is developed that successively refines the initial predictions towards the exact system solutions. The application of POD-2G as a standalone solver or as preconditioner in the context of preconditioned conjugate gradient methods is demonstrated on several numerical examples of large scale systems, with the results indicating its strong superiority over conventional iterative solution schemes.

show abstract

Section: Surrogate Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

AI-enhanced iterative solvers for accelerating the solution of large scale parametrized linear systems of equations

Nikolopoulos

Kalogeris

Papadopoulos

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the sake of generality we denote as W e , b e and W d , b d the weight matrices and bias vectors of the CAEs. A detailed explanation of the CAE architecture is presented in [49].…”

Section: Non-linear Dimensionality Reductionmentioning

confidence: 99%

“…In recent years, NIROMs DL-based have been the subject of several studies aiming to overcome the limitations of the linear projections methods [28], by recovering non-linear, low-dimensional manifolds [36,42]. Various methodologies have been proposed, including supervised and unsupervised DL techniques, to identify low-dimensional manifolds and nonlinear dynamic behaviors [49]. A common methodology applied in the frame of NIROMs DL-based is the convolutional autoencoders (CAE) for the non-linear dimensionality reduction, and the long short-term memory (LSMT) networks to predict the temporal evolution [7,50].…”

Section: Introductionmentioning

confidence: 99%

$\textit{FastSVD-ML-ROM}$: A Reduced-Order Modeling Framework based on Machine Learning for Real-Time Applications

Drakoulas¹,

V.²,

Bourantas³

et al. 2022

Preprint

View full text Add to dashboard Cite

Digital twins have emerged as a key technology for optimizing the performance of engineering products and systems. High-fidelity numerical simulations constitute the backbone of engineering design, providing an accurate insight into the performance of complex systems. However, largescale, dynamic, non-linear models require significant computational resources and are prohibitive for real-time digital twin applications. To this end, reduced order models (ROMs) are employed, to approximate the high-fidelity solutions while accurately capturing the dominant aspects of the physical behavior. The present work proposes a new machine learning (ML) platform for the development of ROMs, to handle large-scale numerical problems dealing with transient nonlinear partial differential equations. Our framework, mentioned as FastSVD-ML-ROM, utilizes (i) a singular value decomposition (SVD) update methodology, to compute a linear subspace of the multi-fidelity solutions during the simulation process, (ii) convolutional autoencoders for nonlinear dimensionality reduction, (iii) feed-forward neural networks to map the input parameters to the latent spaces, and (iv) long short-term memory networks to predict and forecast the dynamics of parametric solutions. The efficiency of the FastSVD-ML-ROM framework is demonstrated for a 2D linear convection-diffusion equation, the problem of fluid around a cylinder, and the 3D blood flow inside an arterial segment. The accuracy of the reconstructed results demonstrates the robustness and assesses the efficiency of the proposed approach.

show abstract

“…In order to optimize end-to-end, the forward model must be differentiable and computationally efficient. When this is not the case, an alternative approach is to train a surrogate neural network, f , to approximate the forward model [38,31,33,34]. However, even well-trained surrogates may result in errors when included in our end-to-end framework, due to the encoders' ability to learn to exploit the surrogate model.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Neural Autoencoder for Sensory Neuroprostheses and Its Applications in Bionic Vision

Granley¹,

Relic²,

Beyeler³

2022

Preprint

View full text Add to dashboard Cite

Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capacities. However, sensations elicited by current devices often appear artificial and distorted. Although current models can often predict the neural or perceptual response to an electrical stimulus, an optimal stimulation strategy solves the inverse problem: what is the required stimulus to produce a desired response? Here we frame this as an end-to-end optimization problem, where a deep neural network encoder is trained to invert a known, fixed forward model that approximates the underlying biological system. As a proof of concept, we demonstrate the effectiveness of our hybrid neural autoencoder (HNA) on the use case of visual neuroprostheses. We found that HNA is able to produce high-fidelity stimuli from the MNIST and COCO datasets that outperform conventional encoding strategies and surrogate techniques across all tested conditions. Overall this is an important step towards the long-standing challenge of restoring high-quality vision to people living with incurable blindness and may prove a promising solution for a variety of neuroprosthetic technologies.Preprint. Under review.

show abstract

Non-intrusive surrogate modeling for parametrized time-dependent partial differential equations using convolutional autoencoders

Cited by 45 publications

References 48 publications

AI-enhanced iterative solvers for accelerating the solution of large scale parametrized linear systems of equations

AI-enhanced iterative solvers for accelerating the solution of large scale parametrized linear systems of equations

$\textit{FastSVD-ML-ROM}$: A Reduced-Order Modeling Framework based on Machine Learning for Real-Time Applications

A Hybrid Neural Autoencoder for Sensory Neuroprostheses and Its Applications in Bionic Vision

Contact Info

Product

Resources

About