Preparing for the Future -- Rethinking Proxy Apps

Abstract

A considerable amount of research and engineering went into designing proxy applications, which represent common high-performance computing workloads, to co-design and evaluate the current generation of supercomputers, e.g., RIKEN's Supercomputer Fugaku, ANL's Aurora, or ORNL's Frontier. This process was necessary to standardize the procurement while avoiding duplicated effort at each HPC center to develop their own benchmarks. Unfortunately, proxy applications force HPC centers and providers (vendors) into a an undesirable state of rigidity, in contrast to the fast-moving trends of current technology and future heterogeneity. To accommodate an extremelyheterogeneous future, we have to reconsider how to co-design supercomputers during the next decade, and avoid repeating the past mistakes. This position paper outlines the current state-of-the-art in system co-design, challenges encountered over the past years, and a proposed plan to move forward.

A BenchCouncil View on Benchmarking Emerging and Future Computing

“…Matsuoka et al also found the implementation of biases and complexity traps [30] in the instantiation process: on the one hand, any implementation of a computer workload benchmark entails multiple implicit biases towards algorithms, programming languages, data layouts, and parallelization approaches; on the other hand, the benchmarks, abstracted from large or legacy scientific codes and tuned for previous computer architectures, trap the co-design participants into considering only similar architectures.…”