A survey of out-of-core algorithms in numerical linear algebra

Abstract

Abstract. This paper surveys algorithms that efficiently solve linear equations or compute eigenvalues even when the matrices involved are too large to fit in the main memory of the computer and must be stored on disks. The paper focuses on scheduling techniques that result in mostly sequential data accesses and in data reuse, and on techniques for transforming algorithms that cannot be effectively scheduled. The survey covers out-of-core algorithms for solving dense systems of linear equations, for the direct and iterative solution of sparse systems, for computing eigenvalues, for fast Fourier transforms, and for N-body computations. The paper also discusses reasonable assumptions on memory size, approaches for the analysis of out-of-core algorithms, and relationships between out-of-core, cache-aware, and parallel algorithms.

Out-of-Core Solution of Eigenproblems for Macromolecular Simulations

“…Our OOC-GPU algorithm for this operation encodes a left-looking, slab-oriented factorization [16] that transfers data by column blocks (slabs) of width s. Note that, while there exist linear algebra libraries to obtain the QR factorization on GPUs [11], these lack of the specialized kernels that are necessary for our particular operation.…”

“…In our QR OOC algorithm, only the orthogonal matrix of the resulting QR factorization of D is built/stored while the upper triangular factor is not referenced/kept. Our QR algorithm is a left-looking algorithm that applies all previous transformations to the current slab -in contrast with the traditional right-looking approach that immediately propagates the transforms to the right of the current slab-since left-looking OOC variants in general incur in a smaller number of transfers [16]. …”

“…To address this, we consider two specialized algorithms that, by applying out-of-core (OOC) techniques [16], amortize the cost of data transfers with a large number of floating-point arithmetic operations (flops). Besides, to deliver high performance, both "OOC-GPU" algorithms off-load the bulk of their computations to the attached hardware graphics accelerator.…”

Solving sequences of generalized least-squares problems on multi-threaded architectures

“…The most common scenarios of GWAS require the processing of data sets that greatly exceed common main memory capacity: in a typical scenario, where 36 millions of GLS problems are to be solved with n = 10,000, the size of the input operand X R is roughly 3 terabytes. To overcome this limitation, we turn our attention to out-of-core algorithms [Toledo 1999]. The goal is to design algorithms that make a proper use of available input/output (I/O) mechanisms to deal with data sets as large as the hard-drive size, while sustaining in-core high performance.…”

Using Graphics Processors to Accelerate the Solution of Out-of-Core Linear Systems

“…• There are only a few Open Source libraries for OOC dense linear algebra operations and most of these target distributed-memory platforms [2,7,8,18,15,17].…”