Designing Bit-Reproducible Portable High-Performance Applications

Arteaga, Andrea; Fuhrer, Oliver; Hoefler, Torsten

doi:10.1109/ipdps.2014.127

Cited by 18 publications

(24 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This richness forces us to adopt an approach that is as generic as possible and consists of compiling different files at different optimizations and drawing a final linked image from this mixture. The granularity of mixing versions in our case is either at a file level, or (by using weak symbols and overriding) at a function level 3 . When we encounter a numerical result difference during our bisection search, we allow existing tools to help with rootcausing.…”

Section: Bisect Problemmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Level Analysis of Compiler-Induced Variability and Performance Tradeoffs

Bentley

Briggs

Gopalakrishnan

et al. 2019

Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing

View full text Add to dashboard Cite

No Yes No Yes Create FLiT tests User Code Deterministic? Reproducibility and Performance Is the fastest repro sufficient? Done Library, Source, and Function Blame Debug Issue using standard tools Determinize Run FLiT Tests FLiT Bisect Done Figure 1: Multi-level workflow. Levels are (1) determine variability-inducing compilations, (2) analyze the space of reproducibility and performance, and (3) debug variability by identifying files and functions causing variability. ABSTRACTSuccessful HPC software applications are long-lived. When ported across machines and their compilers, these applications often produce different numerical results, many of which are unacceptable. Such variability is also a concern while optimizing the code more aggressively to gain performance. Efficient tools that help locate the program units (files and functions) within which most of the variability occurs are badly needed, both to plan for code ports and to root-cause errors due to variability when they happen in the field. In this work, we offer an enhanced version of the open-source testing framework FLiT to serve these roles. Key new features of FLiT include a suite of bisection algorithms that help locate the root causes of variability. Another added feature allows an analysis of the tradeoffs between performance and the degree of variability. Our new contributions also include a collection of case studies. Results on the MFEM finite-element library include variability/performance tradeoffs, and the identification of a (hitherto unknown) abnormal level of result-variability even under mild compiler optimizations. Results from studying the Laghos proxy application include identifying a significantly divergent floating-point result-variability and successful root-causing down to the problematic function over as little as 14 program executions. Finally, in an evaluation of 4,376 controlled injections of floating-point perturbations on the LULESH proxy application, we showed that the FLiT framework has 100% precision and recall in discovering the file and function locations of the injections all within an average of only 15 program executions.

show abstract

Section: Bisect Problemmentioning

confidence: 99%

“…There is little external control one can exert on GPU warp schedulers 3. The approach of searching by overriding symbols is one that potentially creates "Frankenbinaries. "…”

mentioning

confidence: 99%

Multi-Level Analysis of Compiler-Induced Variability and Performance Tradeoffs

Bentley

Briggs

Gopalakrishnan

et al. 2019

Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing

View full text Add to dashboard Cite

show abstract

“…6 Sequential Reproducible Summation [22], (referred as ReproBLAS) from the ReproBLAS library 6 ; 5. The single-sweep reduction [23] with two and three levels (cited as bitrep2 and bitrep3, accordingly) from the bitrep library 7 .…”

Section: Baseline Algorithms and Experimental Setupmentioning

confidence: 99%

“…Demmel and Nguyen have extended their concept to reproducible BLAS routines, distributed in their ReproBLAS library. Arteaga et al [23] used Demmel's and Nguyen's algorithms with improved communication between nodes based on both one-and two-reductions. They were able to reach the same accuracy with better performance results (the overhead is within 10 %) compared to the conventional summation in case of the single reduction algorithm.…”

Section: Related Workmentioning

confidence: 99%

Numerical reproducibility for the parallel reduction on multi- and many-core architectures

et al. 2015

View full text Add to dashboard Cite

Abstract. On modern multi-core, many-core, and heterogeneous architectures, floating-point computations, especially reductions, may become non-deterministic and, therefore, non-reproducible mainly due to the nonassociativity of floating-point operations. We introduce an approach to compute the correctly rounded sums of large floating-point vectors accurately and efficiently, achieving deterministic results by construction. Our multi-level algorithm consists of two main stages: first, a filtering stage that relies on fast vectorized floating-point expansion; second, an accumulation stage based on superaccumulators in a high-radix carry-save representation. We present implementations on recent Intel desktop and server processors, Intel Xeon Phi co-processors, and both AMD and NVIDIA GPUs. We show that numerical reproducibility and bit-perfect accuracy can be achieved at no additional cost for large sums that have dynamic ranges of up to 90 orders of magnitude by leveraging arithmetic units that are left underused by standard reduction algorithms.

show abstract

“…The remaining functions offered by the Oracle database (see https://docs.oracle.com/database/121/SQLRF/functions003.htm) can be computed with SUM. 3 If arithmetic expressions are computed in their entirety only once all operands are available, then executing them always in the same order is trivial. However, if expressions are broken up and partially pushed through joins, then their execution order may depend on the join order, which may change even if the logical input has not changed.…”

Section: Introductionmentioning

confidence: 99%

Reproducible Floating-Point Aggregation in RDBMSs

Müller

Arteaga

Hoefler

et al. 2018

2018 IEEE 34th International Conference on Data Engineering (ICDE)

Self Cite

View full text Add to dashboard Cite

Industry-grade database systems are expected to produce the same result if the same query is repeatedly run on the same input. However, the numerous sources of non-determinism in modern systems make reproducible results difficult to achieve. This is particularly true if floating-point numbers are involved, where the order of the operations affects the final result.As part of a larger effort to extend database engines with data representations more suitable for machine learning and scientific applications, in this paper we explore the problem of making relational GROUPBY over floating-point formats bit-reproducible, i.e., ensuring any execution of the operator produces the same result up to every single bit. To that aim, we first propose a numeric data type that can be used as drop-in replacement for other number formats and is-unlike standard floating-point formats-associative. We use this data type to make state-of-theart GROUPBY operators reproducible, but this approach incurs a slowdown between 4 × and 12 × compared to the same operator using conventional database number formats. We thus explore how to modify existing GROUPBY algorithms to make them bitreproducible and efficient. By using vectorized summation on batches and carefully balancing batch size, cache footprint, and preprocessing costs, we are able to reduce the slowdown due to reproducibility to a factor between 1.9 × and 2.4 × of aggregation in isolation and to a mere 2.7 % of end-to-end query performance even on aggregation-intensive queries in MonetDB. We thereby provide a solid basis for supporting more reproducible operations directly in relational engines.This document is an extended version of an article currently in print for the proceedings of ICDE'18 with the same title and by the same authors. The main additions are more implementation details and experiments.

show abstract

Designing Bit-Reproducible Portable High-Performance Applications

Cited by 18 publications

References 16 publications

Multi-Level Analysis of Compiler-Induced Variability and Performance Tradeoffs

Multi-Level Analysis of Compiler-Induced Variability and Performance Tradeoffs

Numerical reproducibility for the parallel reduction on multi- and many-core architectures

Reproducible Floating-Point Aggregation in RDBMSs

Contact Info

Product

Resources

About