Architectural Constraints to Attain 1 Exaflop/s for Three Scientific Application Classes

In the generalized minimal residual method (GMRES), the global all-to-all communication required in each iteration for orthogonalization and normalization of the Krylov base vectors is becoming a performance bottleneck on massively parallel machines. Long latencies, system noise, and load imbalance cause these global reductions to become very costly global synchronizations. In this work, we propose the use of nonblocking or asynchronous global reductions to hide these global communication latencies by overlapping them with other communications and calculations. A pipelined variation of GMRES is presented in which the result of a global reduction is used only one or more iterations after the communication phase has started. This way, global synchronization is relaxed and scalability is much improved at the expense of some extra computations. The numerical instabilities that inevitably arise due to the typical monomial basis by powering the matrix are reduced and often annihilated by using Newton or Chebyshev bases instead. Our parallel experiments on a medium-sized cluster show significant speedups of the pipelined solvers compared to standard GMRES. An analytical model is used to extrapolate the performance to future exascale systems.

show abstract

“…As the number of nodes increases, the global reductions required in lines 4 and 6 may well become the bottleneck [2,14,5] …”

Section: Standard Gmresmentioning

confidence: 99%

“…5 The full system has 1,202 compute nodes (9,984 processor cores) with a theoretical peak performance of 106 Tflops/sec. All nodes are interconnected by 4×QDR InfiniBand technology, providing 32 Gb/s of point-to-point bandwidth for high-performance message passing and I/O.…”

Section: Timings On Carvermentioning

confidence: 99%

Hiding Global Communication Latency in the GMRES Algorithm on Massively Parallel Machines

Ghysels¹,

Ashby²,

Meerbergen³

et al. 2013

SIAM J. Sci. Comput.

102

134

View full text Add to dashboard Cite

show abstract

“…Bhatele et al [11] presented a feasibility study for three application classes in order to formulate the constraints for achieving a sustained performance of 1 Exaflop/s. Their work is important in terms of providing the possible application classes (molecular dynamics, cosmological N-body simulations, etc.)…”

Section: Related Studiesmentioning

confidence: 99%

“…This paper is helpful, but it covers different aspects (i.e., comparing different vendors for energy usage and performance). However, our goal is to focus on a few vendors and provide a comprehensive evaluation of HPC benchmarking using application kernels that are widely accepted in the HPC community [11,9].…”

Section: Related Studiesmentioning

confidence: 99%

Evaluating ARM HPC clusters for scientific workloads

Maqbool

Fox

2015

Concurrency and Computation

View full text Add to dashboard Cite

The power consumption of modern high-performance computing (HPC) systems that are built using power hungry commodity servers is one of the major hurdles for achieving Exascale computation. Several efforts have been made by the HPC community to encourage the use of low-powered system-on-chip (SoC) embedded processors in large-scale HPC systems. These initiatives have successfully demonstrated the use of ARM SoCs in HPC systems, but there is still a need to analyze the viability of these systems for HPC platforms before a case can be made for Exascale computation. The major shortcomings of current ARM-HPC evaluations include a lack of detailed insights about performance levels on distributed multicore systems and performance levels for benchmarking in large-scale applications running on HPC. In this paper, we present a comprehensive evaluation of results that covers major aspects of server and HPC benchmarking for ARMbased SoCs. For the experiments, we built an unconventional cluster of ARM Cortex-A9s that is referred to as Weiser and ran single-node benchmarks (STREAM, Sysbench, and PARSEC) and multi-node scientific benchmarks (High-performance Linpack (HPL), NASA Advanced Supercomputing (NAS) Parallel Benchmark, and Gadget-2) in order to provide a baseline for performance limitations of the system. Based on the experimental results, we claim that the performance of ARM SoCs depends heavily on the memory bandwidth, network latency, application class, workload type, and support for compiler optimizations. During server-based benchmarking, we observed that when performing memory intensive benchmarks for database transactions, x86 performed 12% better for multithreaded query processing. However, ARM performed four times better for performance to power ratios for a single core and 2.6 times better on four cores. We noticed that emulated double precision floating point in Java resulted in three to four times slower performance as compared with the performance in C for CPU-bound benchmarks. Even though Intel x86 performed slightly better in computation-oriented applications, ARM showed better scalability in I/O bound applications for shared memory benchmarks. We incorporated the support for ARM in the MPJ-Express runtime and performed comparative analysis of two widely used message passing libraries. We obtained similar results for network bandwidth, large-scale application scaling, floating-point performance, and energy-efficiency for clusters in message passing evaluations (NBP and Gadget 2 with MPJ-Express and MPICH). Our findings can be used to evaluate the energy efficiency of ARM-based clusters for server workloads and scientific workloads and to provide a guideline for building energy-efficient HPC clusters.

show abstract

“…[9]. Many of these approaches are often limited in their predictive accuracy, and have not been through an extensive validation process on current systems.…”

Section: Related Workmentioning

confidence: 99%

An early performance analysis of POWER7-IH HPC systems

Barker

Hoisie

Kerbyson

2011

Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

In this work we present a performance evaluation of the POWER7-IH processor and of integrated systems built from it. We describe the architecture of P7-IH with an emphasis on those characteristics that have a direct impact on the performance for large-scale HPC systems and applications. An important area of emphasis is the memory and communication subsystems and their impact on achievable application performance. The results from a set of micro-benchmarks are presented that include memory, communication and OS-noise characteristics. In addition the results from several production level applications are analyzed and their performance linked to the results of the micro-benchmarks through the use of accurate performance models. The models will also be employed in exploring the achievable performance of these applications on much larger systems.

show abstract

Architectural Constraints to Attain 1 Exaflop/s for Three Scientific Application Classes

Cited by 15 publications

References 22 publications

Hiding Global Communication Latency in the GMRES Algorithm on Massively Parallel Machines

Hiding Global Communication Latency in the GMRES Algorithm on Massively Parallel Machines

Evaluating ARM HPC clusters for scientific workloads

An early performance analysis of POWER7-IH HPC systems

Contact Info

Product

Resources

About