Asynchronous Task-Based Execution of the Reverse Time Migration for the Oil and Gas Industry

Abstract

We propose a new framework for deploying Reverse Time Migration (RTM) simulations on distributed-memory systems equipped with multiple GPUs. Our software, TB-RTM, infrastructure engine relies on the STARPU dynamic runtime system to orchestrate the asynchronous scheduling of RTM computational tasks on the underlying resources. Besides dealing with the challenging hardware heterogeneity, TB-RTM supports tasks with different workload characteristics, which stress disparate components of the hardware system. RTM is challenging in that it operates intensively at both ends of the memory hierarchy, with compute kernels running at the highest level of the memory system, possibly in GPU main memory, while I/O kernels are saving solution data to fast storage. We consider how to span the wide performance gap between the two extreme ends of the memory system, i.e., GPU memory and fast storage, on which large-scale RTM simulations routinely execute. To maximize hardware occupancy while maintaining high memory bandwidth throughout the memory subsystem, our framework presents the new out-of-core (OOC) feature from STARPU to prefetch data solutions in and out not only from/to the GPU/CPU main memory but also from/to the fast storage system. The OOC technique may trigger opportunities for overlapping expensive data movement with computations. TB-RTM framework addresses this challenging problem of heterogeneity with a systematic approach that is oblivious to the targeted hardware architectures. Our resulting RTM framework can effectively be deployed on massively parallel GPU-based systems, while delivering performance scalability up to 500 GPUs.

Hierarchical algorithms on hierarchical architectures

“…If DRAM is exhausted, out-of-core approaches may be considered. The StarPU dynamic runtime system provides support for out-of-core algorithms that can mitigate data motion overhead [21]. TLR is attractive because it can be retrofit into existing tile-based shared-memory and distributed-memory software by simply overloading the fully dense matrix kernel operations with their low-rank counterparts.…”

“…Taking this matter into account with a heuristically derived process grid [20], we have factored a TLR covariance matrix of dimension 42 million (representing the covariances of this many points in a 3D cube) in less than 24 h. Notwithstanding that the dense covariance matrix would not fit in the available DRAM memory, if we extrapolate its solution time using the same peak computational rate of approximately 3. is exhausted, out-of-core approaches may be considered. The StarPU dynamic runtime system provides support for out-of-core algorithms that can mitigate data motion overhead [21].…”