A General Approach to Seismic Inversion with Automatic Differentiation

Zhu, Weiqiang; Xu, Kailai; Darve, Eric; Beroza, Gregory C.

doi:10.48550/arxiv.2003.06027

Cited by 3 publications

(5 citation statements)

References 53 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this example, we consider the acoustic wave equation with perfectly matched layer (PML) [46,17]. The governing equation for the acoustic equation is…”

Section: Acoustic Wave Equationmentioning

confidence: 99%

“…However, as the problem size grows, the memory consumption becomes prohibitive because reverse-mode AD requires saving all intermediate results. For example, a direct AD implementation of a 2D elastic equation double-precision solver with a mesh size 1000×1000 and 2000 steps requires at least 193 gigabytes memory 2 [17], which is impractical for a typical CPU. Parallel computing using MPI [18,19] is the de-facto standard for solving such a large-scale problem on modern distributed memory high performance computing (HPC) architectures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Machine Learning for Computational Engineering using MPI

Xu,

Zhu,

Darve

2020

Preprint

Self Cite

View full text Add to dashboard Cite

We propose a framework for training neural networks that are coupled with partial differential equations (PDEs) in a parallel computing environment. Unlike most distributed computing frameworks for deep neural networks, our focus is to parallelize both numerical solvers and deep neural networks in forward and adjoint computations. Our parallel computing model views data communication as a node in the computational graph for numerical simulations. The advantage of our model is that data communication and computing are cleanly separated and thus provide better flexibility, modularity, and testability. We demonstrate using various large-scale problems that we can achieve substantial acceleration by using parallel solvers for PDEs in training deep neural networks that are coupled with PDEs.

show abstract

“…In this example, we consider the acoustic wave equation with perfectly matched layer (PML) [46,17]. The governing equation for the acoustic equation is…”

Section: Acoustic Wave Equationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Distributed Machine Learning for Computational Engineering using MPI

Xu,

Zhu,

Darve

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A promising direction is to combine neural networks and PDEs to formulate FWI as a physics-constrained machine learning problem. Richardson (2018) and W. Zhu et al (2020) have implemented FWI using deep learning frameworks so that reverse-mode automatic differentiation and the various effective optimization tools in deep learning frameworks can be used for FWI. These studies demonstrate the similarity between FWI and neural networks.…”

Section: Introductionmentioning

confidence: 99%

“…We calculate the gradients of both the generative neural network and the PDEs by automatic differentiation, which simplifies the optimization with both neural networks and PDEs. Because adjoint-state methods and reverse-mode automatic differentiation are mathematically equivalent (W. Zhu et al, 2020), we can also use adjoint-state methods for the optimization of NNFWI. NNFWI is implemented based on ADCME 1 and ADSeismic 2 .…”

Section: Introductionmentioning

confidence: 99%

Integrating Deep Neural Networks with Full-waveform Inversion: Reparametrization, Regularization, and Uncertainty Quantification

Zhu,

Xu,

Darve

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Full-waveform inversion (FWI) is an imaging approach for modeling velocity structure by minimizing the misfit between recorded and predicted seismic waveforms. The strong nonlinearity of FWI resulting from fitting oscillatory waveforms can trap the optimization in local minima. We propose a neural-network-based full waveform inversion method (NNFWI) that integrates deep neural networks with FWI by representing the velocity model with a generative neural network. Neural networks can naturally introduce spatial correlations as regularization to the generated velocity model, which suppresses noise in the gradients and mitigates local minima. The velocity model generated by neural networks is input to the same partial differential equation (PDE) solvers used in conventional FWI. The gradients of both the neural networks and PDEs are calculated using automatic differentiation, which back-propagates gradients through the acoustic/elastic PDEs and neural network layers to update the weights and biases of the generative neural network. Experiments on 1D velocity models, the Marmousi model, and the 2004 BP model demonstrate that NNFWI can mitigate local minima, especially for imaging high contrast features like salt bodies, and significantly improves the inversion in the presence of noise. Adding dropout layers to the neural network model also allows analyzing the uncertainty of the inversion results through Monte Carlo dropout. NNFWI opens a new pathway to combine deep learning and FWI for exploiting both the characteristics of deep neural networks and the high accuracy of PDE solvers. Because NNFWI does not require extra training data and optimization loops, it provides an attractive and straightforward alternative to conventional FWI.

show abstract

“…The inversion problem is a very general subject of investigation, studied in many different fields such as medical physics [4] or seismic inversion [5], and recently deep learning has shown to be a promising approach [6][7][8][9] for its solution, taking advantage of the computational advances made possible by the availability of graphical processing units (GPU). The method we adopt is based on creating a database of simulated luminosity distance data obtained by solving the direct problem for a large set or random density and velocity configurations.…”

mentioning

confidence: 99%

Deep learning reconstruction of the large scale structure of the Universe from luminosity distance observations

García¹,

Santa²,

Romano³

2021

Preprint

View full text Add to dashboard Cite

Supernovae Ia (SN) are among the brightest objects we can observe and can provide a unique window on the large scale structure of the Universe at redshifts where other observations are not available. The photons emitted by SNe are in fact affected by the density field between the source and the observer, and from the observed luminosity distance it is possible to solve the inversion problem (IP), i.e. to reconstruct the density field which produced those effects.So far the IP was only solved assuming some restrictions about the geometry of the problem, such as spherical symmetry for example, and the approach was based on solving complicated systems of differential equations which required smooth function as inputs, while observational data is not smooth, due to its discrete nature. In order to overcome these limitations we develop for the first time an inversion method which is not assuming any symmetry, and can be applied directly to observational data, without the need of any data smoothing procedure.The method is based on the use of convolutional neural networks (CNN) trained on simulated data, and it shows quite accurate results. The training data set is obtained by first generating random density and velocity profiles, and then computing their effects on the luminosity distance. The CNN is then trained to reconstruct the density field from the luminosity distance. The CNN is a modified version of U-Net to account for the tridimensionality of the data, and can reconstruct the density and velocity fields with a good level of accuracy.The use of neural networks to analyze observational data from future SNe catalogues will allow to reconstruct the large scale structure of the Universe to an unprecedented level of accuracy, at a redshift at which few other observations are available.

show abstract

A General Approach to Seismic Inversion with Automatic Differentiation

Cited by 3 publications

References 53 publications

Distributed Machine Learning for Computational Engineering using MPI

Distributed Machine Learning for Computational Engineering using MPI

Integrating Deep Neural Networks with Full-waveform Inversion: Reparametrization, Regularization, and Uncertainty Quantification

Deep learning reconstruction of the large scale structure of the Universe from luminosity distance observations

Contact Info

Product

Resources

About