Architecture and Performance of Devito, a System for Automated Stencil Computation

Luporini, Fabio; Louboutin, Mathias; Lange, Michael; Kukreja, Navjot; Witte, Patrick; Hückelheim, Jan; Yount, Charles; Kelly, Paul H. J.; Herrmann, Felix J.; Gorman, Gerard J.

doi:10.1145/3374916

Cited by 97 publications

(78 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to earlier work, addressing the memory demands of (LS‐)RTM via on‐fly‐Fourier transforms (Witte et al ., 2019c), we tackle the problem of on‐the‐fly source estimation in the time domain. Because we use industry‐strength time‐domain finite‐difference propagators provided by Devito (Luporini et al ., 2018; Louboutin et al ., 2019) and exposed in the Julia programming language by JUDI (Witte et al ., 2019b), our approach scales in principle to large 3D industrial problems. While we stressed the importance of estimating the source function, we believe that the sensitivity of least‐squares RTM (LS‐RTM) to errors in the background velocity model needs to be studied more carefully albeit early work on time‐harmonic LS‐RTM shows robustness with respect to these errors (Tu and Herrmann, 2015a).…”

Section: Discussionmentioning

confidence: 99%

“…To keep our time‐domain wave‐equation solvers with finite differences numerically stable (in our implementation, we used Devito, https://www.devitoproject.org) for our time‐domain finite‐difference simulations and gradient computations (Luporini et al ., 2018; Louboutin et al ., 2019) and JUDI (https://github.com/slimgroup/JUDI.jl) as an abstract linear algebra interface to our algorithms (Witte et al ., 2019b), we introduce an initial guess for the source‐time function

q_{0}

with a bandwidth‐limited spectrum that is flat over the frequency range of interest. Under some assumptions on the source‐time function, we can write the true source‐time function as the convolution between the initial guess and the unknown filter

boldw

– that is, we have

q = w * {boldq}_{0}

where the symbol

*

denotes the temporal convolution.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Time‐domain sparsity promoting least‐squares reverse time migration with source estimation

Yang

Fang

Witte

et al. 2020

Geophysical Prospecting

Self Cite

View full text Add to dashboard Cite

Least-squares reverse-time migration is well known for its capability to generate artefact-free true-amplitude subsurface images through fitting observed data in the least-squares sense. However, when applied to realistic imaging problems, this approach is faced with issues related to overfitting and excessive computational costs induced by many wave-equation solves. The fact that the source function is unknown complicates this situation even further. Motivated by recent results in stochastic optimization and transform-domain sparsity promotion, we demonstrate that the computational costs of inversion can be reduced significantly while avoiding imaging artefacts and restoring amplitudes. While powerful, these new approaches do require accurate information on the source-time function, which is often lacking. Without this information, the imaging quality deteriorates rapidly. We address this issue by presenting an approach where the source-time function is estimated on the fly through a technique known as variable projection. Aside from introducing negligible computational overhead, the proposed method is shown to perform well on imaging problems with noisy data and problems that involve complex settings such as salt. In either case, the presented method produces high-resolution high-amplitude fidelity images including an estimate for the source-time function. In addition, due to its use of stochastic optimization, we arrive at these images at roughly one to two times the cost of conventional reverse-time migration involving all data.

show abstract

Section: Discussionmentioning

confidence: 99%

q_{0}

boldw

– that is, we have

q = w * {boldq}_{0}

where the symbol

*

denotes the temporal convolution.…”

Section: Methodsmentioning

confidence: 99%

Time‐domain sparsity promoting least‐squares reverse time migration with source estimation

Yang

Fang

Witte

et al. 2020

Geophysical Prospecting

Self Cite

View full text Add to dashboard Cite

show abstract

“…Reference Problem We use Devito (Kukreja et al, 2016;Luporini et al, 2018) to build an acoustic wave propagation experiment. The velocity model was initialized using the SEG Overthrust model.…”

Section: Methodsmentioning

confidence: 99%

Lossy Checkpoint Compression in Full Waveform Inversion

Kukreja¹,

Hückelheim²,

Louboutin³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Abstract. This paper proposes a new method that combines check- pointing methods with error-controlled lossy compression for large-scale high-performance Full-Waveform Inversion (FWI), an inverse problem commonly used in geophysical exploration. This combination can signif- icantly reduce data movement, allowing a reduction in run time as well as peak memory. In the Exascale computing era, frequent data transfer (e.g., memory bandwidth, PCIe bandwidth for GPUs, or network) is the performance bottleneck rather than the peak FLOPS of the processing unit. Like many other adjoint-based optimization problems, FWI is costly in terms of the number of floating-point operations, large memory foot- print during backpropagation, and data transfer overheads. Past work for adjoint methods has developed checkpointing methods that reduce the peak memory requirements during backpropagation at the cost of additional floating-point computations. Combining this traditional checkpointing with error-controlled lossy compression, we explore the three-way tradeoff between memory, precision, and time to solution. We investigate how approximation errors introduced by lossy compression of the forward solution impact the objective function gradient and final inverted solution. Empirical results from these numerical experiments indicate that high lossy-compression rates (compression factors ranging up to 100) have a relatively minor impact on convergence rates and the quality of the final solution.

show abstract

“…There have been few attempts to provide a modular framework for the stencil computations. Devito [29], [30] generates optimized wave propagation kernels for use in seismic imaging. However, Devito relies on the fork-join paradigm from the OpenMP programming model [31] and SIMD optimizations for parallel performance.…”

Section: Related Workmentioning

confidence: 99%

Asynchronous Task-Based Execution of the Reverse Time Migration for the Oil and Gas Industry

AlOnazi

Ltaief

Keyes

et al. 2019

2019 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

We propose a new framework for deploying Reverse Time Migration (RTM) simulations on distributed-memory systems equipped with multiple GPUs. Our software, TB-RTM, infrastructure engine relies on the STARPU dynamic runtime system to orchestrate the asynchronous scheduling of RTM computational tasks on the underlying resources. Besides dealing with the challenging hardware heterogeneity, TB-RTM supports tasks with different workload characteristics, which stress disparate components of the hardware system. RTM is challenging in that it operates intensively at both ends of the memory hierarchy, with compute kernels running at the highest level of the memory system, possibly in GPU main memory, while I/O kernels are saving solution data to fast storage. We consider how to span the wide performance gap between the two extreme ends of the memory system, i.e., GPU memory and fast storage, on which large-scale RTM simulations routinely execute. To maximize hardware occupancy while maintaining high memory bandwidth throughout the memory subsystem, our framework presents the new out-of-core (OOC) feature from STARPU to prefetch data solutions in and out not only from/to the GPU/CPU main memory but also from/to the fast storage system. The OOC technique may trigger opportunities for overlapping expensive data movement with computations. TB-RTM framework addresses this challenging problem of heterogeneity with a systematic approach that is oblivious to the targeted hardware architectures. Our resulting RTM framework can effectively be deployed on massively parallel GPU-based systems, while delivering performance scalability up to 500 GPUs.

show abstract

Architecture and Performance of Devito, a System for Automated Stencil Computation

Cited by 97 publications

References 23 publications

Time‐domain sparsity promoting least‐squares reverse time migration with source estimation

Time‐domain sparsity promoting least‐squares reverse time migration with source estimation

Lossy Checkpoint Compression in Full Waveform Inversion

Asynchronous Task-Based Execution of the Reverse Time Migration for the Oil and Gas Industry

Contact Info

Product

Resources

About