We describe a multistage parallel linear solver framework developed as part of the Intersect (IX) next-generation reservoir simulation project. The object-oriented framework allows for wide flexibility in the number of stages, methods and preconditioners. Here, we describe the specific components of a two-stage CPR[1] (Constraint Pressure Residual) scheme designed for large-scale parallel, structured and unstructured linear systems. We developed a highly efficient in-house Parallel Algebraic Multigrid (PAMG) solver as the first stage preconditioner. For the second stage, we use a parallel ILU-type scheme. This new and powerful combination of CPR and PAMG was the result of detailed analysis of the linear system of equations associated with reservoir simulation.
Using several difficult reservoir simulation problems, we demonstrate the robustness and excellent parallel scalability of the IX linear solver. For the field case studies, the IX linear solver with CPR and PAMG is at least five times faster than an established and widely used industrial linear solver. The performance advantage of the IX linear solver over traditional reservoir simulation linear solvers increases with both problem size and the number of processors.
Introduction
Different types of grid may be used for reservoir flow simulation to model geometrically complex, highly detailed models and/or deviated or multi-lateral wells[2]. Grid types are often labeled based on their structure. Examples of simulation grids include:structured Cartesian,structured stratigraphic,multi-block stratigraphic,PEBI (Perpendicular Bisector), andgenerally unstructured. Hybrid grids that combine various types can also be used.
It is now widely recognized that complete flexibility in representing complex and highly detailed simulation models can be achieved using generally unstructured grids[3].In recent years, significant efforts have focused on building multi-purpose reservoir flow simulators that can deal with geometrically complex and highly detailed structured and unstructured reservoir models[4,5,6].
These relatively large-scale efforts are being pursued because, for nearly three decades, the reservoir simulation community has focused on building robust and efficient reservoir simulators for structured grid problems. Today, the ability to routinely simulate a wide spectrum of practical black-oil problems on (effectively) structured models with O(105) gridblocks is widespread. However, the performance of traditional reservoir simulators typically deteriorates significantly with problem size and the number of processors. This is because the algorithms and software implementations were not designed for scalable, parallel computation. A scalable algorithm is one whose computational complexity (i.e., the number of operations to reach solution) is proportional to the number of unknowns; moreover, the algorithm should also have a convergence rate that is independent of problem size or the number of processors. In numerical solution algorithms, there is often a tradeoff between convergence rate and degree of parallelism. As a result, to obtain a useful measure of parallel efficiency, the best scalar (uniprocessor) algorithm should be used as reference. Scalable methods are needed because the size of problems of interest continues to grow quite significantly, and we want to avoid methods with computational complexities of O(Na) with an athat is (much) larger than unity.
