A Bridging Model for Multi-core Computing

Abstract

Writing software for one parallel system is a feasible though arduous task. Reusing the substantial intellectual effort so expended for programming a second system has proved much more challenging. In sequential computing algorithms textbooks and portable software are resources that enable software systems to be written that are efficiently portable across changing hardware platforms. These resources are currently lacking in the area of multi-core architectures, where a programmer seeking high performance has no comparable opportunity to build on the intellectual efforts of others.In order to address this problem we propose a bridging model aimed at capturing the most basic resource parameters of multi-core architectures. We suggest that the considerable intellectual effort needed for designing efficient algorithms for such architectures may be most fruitfully expended in designing portable algorithms, once and for all, for such a bridging model. Portable algorithms would contain efficient designs for all reasonable combinations of the basic resource parameters and input sizes, and would form the basis for implementation or compilation for particular machines.Our Multi-BSP model is a multi-level model that has explicit parameters for processor numbers, memory/cache sizes, communication costs, and synchronization costs. The lowest level corresponds to shared memory or the PRAM, acknowledging the relevance of that model for whatever limitations on memory and processor numbers it may be efficacious to emulate it.We propose parameter-aware portable algorithms that run efficiently on * This work was supported in part by NSF-CCF-04-27129. A preliminary version appeared in Proc. 16th Annual European Symposium on Algorithms, Sept 15-17, 2008, Karlsruhe Germany, LNCS Vol 5193, pp. 13-28, and had also been presented at SPAA, June [14][15][16] 2008.Email addresses: valiant@seas.harvard.edu (Leslie G. Valiant) January 15, 2010 all relevant architectures with any number of levels and any combination of parameters. For these algorithms we define a parameter-free notion of optimality. We show that for several fundamental problems, including standard matrix multiplication, the Fast Fourier Transform, and comparison sorting, there exist optimal portable algorithms in that sense, for all combinations of machine parameters. Thus some algorithmic generality and elegance can be found in this many parameter setting. Preprint submitted to Elsevier

Low depth cache-oblivious algorithms

“…One or more cache lines of a given cache at level i < k fit precisely in a cache line of its "parent" cache at level i + 1. Tree of caches models [3,4,12,34,37,39] reflect current and future trends in many-core cache hierarchies [31,34,37], with increasing numbers of cores and new memory technologies (Flash, PCM, etc.) increasing the numbers of leaves and levels in the hierarchy.…”

Parallel Programming Applied Research Projects for Teaching Parallel Programming to Beginner Students

“…AND MSC-LEVEL TEACHING The BSP (bulk-synchronous parallel) model of parallel computing [36], [37], [38] has been studied, applied and implemented since the late 1990s to improve the reliability, predictability and portable scalability of parallel programs.…”