Exploiting Capabilities of Many Core Platforms in Reservoir Simulation

Abstract

The forthcoming generation of many-core architectures suggests a strong paradigm shift in the way algorithms have been designed to achieve maximum performance in reservoir simulations. In this work, we propose a novel poly-algorithmic solver approach to develop hybrid CPU multicore and GPU computations for solving large sparse linear systems arising in realistic black oil and compositional flow scenarios. The GPU implementation exploits data parallelism through the simultaneous deployment of thousands of threads while reducing memory overhead per floating point operations involved in most BLAS kernels and in a suite of preconditioner options such as BILU(k), BILUT and multicoloring SSOR. On the other hand, multicore CPU computations are used to exploit functional parallelism to perform system partitioning and reordering, algebraic multigrid preconditioning, sparsification and model reduction operations in order to accelerate and reduce the number of GCR iterations. The efficient orchestration of these operations relies on carefully designing the heuristics depending on the timestep evolution, degree of nonlinearity and current system properties. Hence, we also propose several criteria to automatically decide the type of solver configuration to be employed at every time step of the simulation. To illustrate the potentials of the proposed solver approach, we perform numerical computations on state-of-the-art multicore CPU and GPU platforms. Computational experiments on a wide range of highly complex reservoir cases reveal that the solver approach yields significant speedups with respect to conventional CPU multicore solver implementations. The solver performance gain is of the order of 3x which impacts in about 2x the overall compositional simulation turnaround time. These results demonstrate the potential that many core solvers have to offer in improving the performance of near future reservoir simulations.

A Parallel Framework for Reservoir Simulators on Distributed-memory Supercomputers

“…These techniques have been applied to black oil simulations [54,30]. Klie and Saad implemented their GPU-based linear solvers and applied them to reservoir simulation [43,45]. Natoli et al developed a GPU-based parallel algebraic multigrid solver and a GPU-based reservoir simulator, which was much faster than existing CPU-based simulators and could handle black oil models with tens of millions of grid cells [20,21,22,23].…”

Accelerating algebraic multigrid solvers on NVIDIA GPUs

“…The Krylov subspace solvers and the incomplete LU factorization (ILU) preconditioners [1][2][3][4] are the most commonly used methods. The ILU methods are efficient and easy to implement.…”

“…The memory speed of GPUs is much higher than memory of CPUs, which makes GPUs good platforms for linear solvers and other scientific computing applications. However, they have different architectures from the traditional CPUs, which means that new implementations must be developed to utilize the power of GPUs, such as FFT [20], BLAS [21,22,20], and Krylov subspace solvers [23,3,4,24]. NVIDIA developed a hybrid matrix format HYB for general matrices [21,22].…”

“…Naumov [25] and Chen et al [24] studied parallel triangular solvers for GPUs. Saad et al developed a type of JAD matrix for GPU computing and its corresponding SpMV algorithm [4,3]. Chen et al designed another hybrid matrix format HEC and SpMV algorithm [26], Krylov solvers [27,28] and classical AMG solver [29].…”

A New Approach to Load Balance for Parallel/Compositional Simulation Based on Reservoir-Model Overdecomposition

“…As graphic processing units (GPUs) have become more and more popular in the oil industry (Foltinek et al 2009;Appleyard et al 2011;Klie et al 2011;Liu et al 2012;Bayat and Killough 2013), it is expected that the reservoir-simulation community will soon have a GPUaccelerated linear solver commercially implemented for reservoir simulation.…”