The Landscape of Exascale Research

Abstract

The next generation of supercomputers will break the exascale barrier. Soon we will have systems capable of at least one quintillion (billion billion) floating-point operations per second (10 18 FLOPS). Tremendous amounts of work have been invested into identifying and overcoming the challenges of the exascale era. In this work, we present an overview of these efforts and provide insight into the important trends, developments, and exciting research opportunities in exascale computing. We use a three-stage approach in which we (1) discuss various exascale landmark studies, (2) use data-driven techniques to analyze the large collection of related literature, and (3) discuss eight research areas in depth based on influential articles. Overall, we observe that great advancements have been made in tackling the two primary exascale challenges: energy efficiency and fault tolerance. However, as we look forward, we still foresee two major concerns: the lack of suitable programming tools and the growing gap between processor performance and data bandwidth (i.e., memory, storage, networks). Although we will certainly reach exascale soon, without additional research, these issues could potentially limit the applicability of exascale computing.

Jet: Fast quantum circuit simulations with parallel task-based tensor-network contraction

“…As high-performance computing (HPC) moves toward increasingly diverse architectures and larger processor counts in the exascale era, new programming models are being developed to mitigate the predicted challenges [10]. The current trend of the high-performance community is a move towards asynchronous task-parallelism [9-11, 28, 29], commonly referred to as asynchronous manytask parallelism.…”

“…Future exascale supercomputers are expected to increase in parallelism and heterogeneity, with billions of threads running concurrently [9,10]. Specifically, the systems will not increase in the number of nodes, but rather in on-node concurrency with large multi-core CPUs and multiple GPUs per node [11][12][13].…”

“…To address issues with current programming models, the high-performance computing community has started considering asynchronous task-parallelism (sometimes referred to as asynchronous many-task run-time models) as a solution [10,14]. Asynchronous task-parallelism is founded on the idea of decomposing an algorithm into units of work, known as tasks, and executing them asynchronously.…”

Innermost many-sorted term rewriting on GPUs

“…GPUs are optimised for the highly parallel and regular computations that occur in graphics processing, but they become more and more interesting for general purpose computations (for instance, see [2][3][4][5][6][7]). It is not without reason that modern super computers have large banks of graphical processors installed in them [8]. GPU designers realise this and make GPUs increasingly suitable for irregular computations.…”

Bayesian Optimization for auto-tuning GPU kernels

“…Graphics Processing Units (GPUs) have revolutionized the computing landscape in the past decade [1], and are seen as one of the enabling factors in recent breakthroughs in Artificial Intelligence [2]. While GPUs are used to enable scientific computing workloads in many fields, including climate modeling, artificial intelligence, and quantum physics, it is actually very hard to unlock the full computational power of the GPU [3].…”