The Cache-Oblivious Gaussian Elimination Paradigm: Theoretical Framework, Parallelization and Experimental Evaluation

Abstract

We consider triply-nested loops of the type that occur in the standard Gaussian elimination algorithm, which we denote by GEP (or the Gaussian Elimination Paradigm). We present two related cache-oblivious methods I-GEP and C-GEP, both of which reduce the number of cache misses incurred (or I/Os performed) by the computation over that performed by standard GEP by a factor of √ M, where M is the size of the cache. Cache-oblivious I-GEP computes in-place and solves most of the known applications of GEP including Gaussian elimination and LU-decomposition without pivoting and Floyd-Warshall all-pairs shortest paths. Cache-oblivious C-GEP uses a modest amount of additional space, but is completely general and applies to any code in GEP form. Both I-GEP and C-GEP produce system-independent cacheefficient code, and are potentially applicable to being used by optimizing compilers for loop transformation.We present parallel I-GEP and C-GEP that achieve good speed-up and match the sequential caching performance cache-obliviously for both shared and distributed caches for sufficiently large inputs.We present extensive experimental results for both in-core and out-of-core performance of our algorithms. We consider both sequential and parallel implementations, and compare them with finely-tuned cache-aware BLAS code for matrix multiplication and Gaussian elimination without pivoting. Our results indicate that cacheoblivious GEP offers an attractive trade-off between efficiency and portability.This work was supported in part by NSF Grant CCF-0514876 and NSF CISE Research Infrastructure Grant EIA-0303609. This journal submission incorporates results on the cache-oblivious paradigm that were presented in preliminary form in [8] and [9].

Competitive Cache Replacement Strategies for Shared Cache Environments

“…In most of these computations, each process also has full knowledge about its future request sequence of its current task. For instance, the computation of Gaussian elimination paradigm as discussed by Chowdhury and Ramachandran [9] has this type of behavior. Even the computation of matrix multiplication and fast Fourier transform have this type of behavior.…”

“…In several of such applications, processes also have perfect knowledge about the sequence of requests they plan to request in the future since they work on a well-defined computation like matrix multiplication or Gaussian elimination paradigm [9,10]. Observe that in these computations, the interleaving of requests from different processes reaching the shared cache still remains adversarial since the interleaving depends on factors like the difference in the clock period, interrupts from the operating systems, etc.…”

Minimizing Communication in All-Pairs Shortest Paths

“…The Gaussian elimination paradigm of Chowdhury and Ramachandran [13] provides a cache-oblivious framework for these problems, similar to Toledo's recursive blocked LU factorization [41]. Our APSP work is orthogonal to that of Chowdhury and Ramachandran in the sense we provide distributed memory algorithms that minimize internode communication (both latency and bandwidth), while their method focuses on cacheobliviousness and multithreaded (shared memory) implementation.…”

Parallel external memory graph algorithms

“…However, their analysis is limited to the hierarchical divide-and-conquer problems and a moderate level of parallelism. Chowdhury and Ramachandran [9] consider cache-complexity in both private-and shared-cache models for matrix-based computations, including all-pairs shortest paths algorithm of FloydWarshall. They also consider parallel dynamic programming algorithms in private-, shared-and multicore-cache models [10].…”