HPC-MixPBench: An HPC Benchmark Suite for Mixed-Precision Analysis

Parasyris, Konstantinos; Laguna, Ignacio; Menon, Harshitha; Schordan, Markus; Osei-Kuffuor, Daniel; Georgakoudis, Giorgis; Lam, Michael O.; Vanderbruggen, Tristan

doi:10.1109/iiswc50251.2020.00012

Cited by 11 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, precision auto-tuning tools aim at providing a mixed precision version of a program that satisfies accuracy requirements, whatever the implemented algorithms. In [3] a benchmark suite of programs is introduced for mixed precision computing analysis. Moreover the authors present the performance of various precision auto-tuning algorithms such as combinational (used in FloatSmith [4]), compositional (used in FloatSmith [4]), Delta Debug (introduced in [5], used in Precimonious [6] and PROMISE [1]), hierarchical (used in CRAFT-HPC [7]), hierarchical-compositional (used in FloatSmith [4]) and a Genetic Search Algorithm (GA) (used in AMPT-GA [8]).…”

Section: Introductionmentioning

confidence: 99%

Performance of precision auto-tuned neural networks

Ferro,

Graillat,

Hilaire

et al. 2023

2023 IEEE 16th International Symposium on Embedded Multicore/Many-Core Systems-on-Chip (MCSoC)

View full text Add to dashboard Cite

While often used in embedded systems, neural networks can be costly in terms of memory and execution time.Reducing the precision used in neural networks can be beneficial in terms of performance and energy consumption. After having applied a floating-point auto-tuning tool, PROMISE, on various neural networks, we obtained versions using lower precision while keeping a required accuracy on the results. In this article, we present results regarding the memory and computation time gains obtained thanks to reduced precision, using vectorized and non-vectorized code. We also show the impact on the execution time of PROMISE of the parallelization of the Delta Debug algorithm it implements.

show abstract

Section: Introductionmentioning

confidence: 99%

Performance of precision auto-tuned neural networks

Ferro,

Graillat,

Hilaire

et al. 2023

2023 IEEE 16th International Symposium on Embedded Multicore/Many-Core Systems-on-Chip (MCSoC)

View full text Add to dashboard Cite

show abstract

“…Its principles preach that although performing the most possible exact computations at scale requires a high amount of computational resources, allowing certain approximations or occasional violations of numerical consistency can provide significant gains in efficiency. [Parasyris et al 2020] states that Approximate Computing exploits the gap between the level of accuracy required by the applications and that provided by the system and has the potential to benefit a wide range of applications, such as scientific computing and machine learning. [Mittal 2015] stands that AC is based on the intuitive observation that while performing the most possible exact computation or maintaining peak-level service requires a high amount of resources, allowing selective approximation or occasional violation of the specification can provide gains in efficiency.…”

Section: Introductionmentioning

confidence: 99%

“…Modern computer architectures sup-port multiple levels of precision for floating-point computations to provide trade-offs between accuracy and performance. Several recent studies, such as [Fogerty et al 2017] and [Parasyris et al 2020], have demonstrated the use of mixed precision, which means using multiple levels of precision, to increase significantly the performance of scientific applications. With accelerators supporting several levels of floating-point precision, such as half, single, and double precision in NVIDIA GPUs, and with higher peak performance in lower precision in these accelerators, this technique has become a promising approach to boost performance, especially using Tensor Cores [Parasyris et al 2020].…”

Section: Introductionmentioning

confidence: 99%

Mixed precision applied on common mathematical procedures over GPU

Sudo

Fazenda²,

Souto³

2022

Anais Do XXIII Simpósio Em Sistemas Computacionais De Alto Desempenho (SSCAD 2022)

View full text Add to dashboard Cite

Approximate Computing is a paradigm used by researchers as alternative to the diminishing of the evolution of hardware performance in the ongoing race for computational throughput in HPC. Precision reduction and mixed precision are the most studied among the existing techniques. In addition, some NVIDIA GPUs have Tensor Core architecture to speed up some classes of algorithms, such as matrix multiplication. This study aims to apply Approximate Computing techniques, like mixed precision, in matrix multiplication and stencil algorithms using OpenACC directives and cuTensor library to analyze performance gains versus accuracy losses. Results showed that it was possible to obtain a speedup of 16.60× with an optimized matrix multiplication algorithm present in the matmul intrinsic function using 16-bit floating-point data with Tensor Core, compared to a naive version using 64-bit floating-point. For this same case, accuracy loss went from 10−26 up to 10−1, approximately. For the stencil algorithm, it was possible to obtain a gain of 1.60× by only reducing variables precision from 64-bit to 16-bit floating-point, with accuracy loss from 0 to 10−9, for 300 iterations.

show abstract