Reformulation of the performance portability metric

Abstract

Summary The 3‐P challenge of high‐performance programming—performance, portability and productivity—has become more difficult than ever in the age of heterogeneous computing. It would be naïve to think that the performance portability problem can be completely solved, but it can certainly be reduced and made tolerable. However, first and foremost, an agreement is needed on what it means for an application to be performance portable. Unfortunately, there is still no consensus in the scientific community on a workable definition of the term performance portability. Several years ago, a comprehensive effort was made to formulate a novel definition of performance portability and an associated metric. Since the new metric was first introduced, it has been widely adopted by the scientific community, and many advanced studies have used it. Unfortunately, the definition of the new metric has flaws. This article presents a proof of the theoretical flaws in the definition of the new metric, considers the practical implications of these flaws as reflected in many studies that have used it in recent years, and proposes a revised metric that addresses the flaws and provides guidelines on how to use it correctly.

Assessing Opportunities of SYCL and Intel oneAPI for Biological Sequence Alignment

“…• Running the SYCL code on other architectures such as AMD CPUs and FPGAs, to extend the cross-architecture portability study. • Carrying out an extensive study of the performance portability of these codes, following Marowka's proposal [35].…”

A novel acceleration method for molecular dynamics of crystal silicon on GPUs using OpenACC

“…20 Ami Marowka points out that the 3P challenges of high-performance programming-performance, portability, and productivity-have become more difficult than ever in the era of heterogeneous computing. 21 Directives strive to offer portability without losing performance and are one of the most portable and productive programming models. 22 Ope-nACC 23 is a directive-based programming model for acceleration devices, which has been widely supported by the industry.…”

Toward a Better Performance Portability Metric

“…to propose the P P metric to calculate the performance portability based on the harmonic mean [7]. But this metric proved itself to be problematic as was articulated in many studies [8], [13], [16]- [18].…”

“…The main claims against the P P metric were that it is unintuitive, unfamiliar, loses information, difficult to use, and the performance portability scores it yields are unrealistic. Therefore, the ¯ PP metric based on the arithmetic mean was proposed, which actually solved the above problems and yielded much more realistic results without losing information [8]. The designers of the P P metric accepted some of the claims but left the rest of the problems unaddressed [11].…”

“…The definition explicitly states that performance efficiency is the ultimate measure of performance portability. Therefore, several approaches were proposed to measure performance efficiency alongside several metrics to calculate performance portability [7], [8], [11]- [13]. And if we add to these facts that there is no agreed framework of rules and guidelines on how to measure and calculate performance efficiency and performance portability, then it is not surprising that it is not possible to draw informed insights from the dozens of studies that have been done in recent years, and it would not be an overstatement to claim that the current situation is a complete mess that can be reorganized.…”