A Probabilistic Approach to Floating-Point Arithmetic

Abstract

Finite-precision floating point arithmetic unavoidably introduces rounding errors which are traditionally bounded using a worst-case analysis. However, worst-case analysis might be overly conservative because worst-case errors can be extremely rare events in practice. Here we develop a probabilistic model of rounding errors with which it becomes possible to estimate the likelihood that the rounding error of an algorithm lies within a given interval. Given an input distribution, we show how to compute the distribution of rounding errors. We do this exactly for low precision arithmetic, for high precision arithmetic we derive a simple approximation. The model is then entirely compositional: given a numerical program written in a simple imperative programming language we can recursively compute the distribution of rounding errors at each step of the computation and propagate it through each program instruction. This is done by applying a formalism originally developed by Kozen to formalize the semantics of probabilistic programs. We then discuss an implementation of the model and use it to perform probabilistic range analyses on some benchmarks. arXiv:1912.00867v2 [math.NA] 10 Dec 2019 Assumption 0. The probability density function is constant at the scale of the intervals between floating point numbers, i.e.for all values of t such that Eq. (8) holds. Assumption 1. Writing z(s, k) for z(e min , s, k) assume that: 0≤k<2 p s∈{0,1}

Rigorous Roundoff Error Analysis of Probabilistic Floating-Point Computations

“…However in order to be sound, we must in general include these three discrete components to our computations. The density dist c is given explicitly by the following result whose proof can already be found in [9]. Theorem 1.…”

“…If one had to choose 'the' canonical distribution for roundoff errors, we claim that the density given below should be this distribution, and we therefore call it the typical distribution; we depict it in Fig. 2 and formalize it with the following theorem, which can mostly be found in [9]. Theorem 3.…”

“…xut dP (9) but it is necessary to use some more advanced probability theory. We will make the simplifying assumption that the density of X is constant on each interval z, z in order to keep the proof manageable.…”

“…For this reason, the famous critique of the probabilistic approach to roundoff errors [29] does not apply to this work. Our preliminary report [9] presents some early ideas behind this work, including Eqs. ( 5) and ( 7) and a very rudimentary range analysis.…”

Deterministic and Probabilistic Error Bounds for Floating Point Summation Algorithms

“…Shifted summation is motivated by work in computer architecture [5,4] and formal methods for program verification [17] where not only the roundoffs but also the inputs are interpreted as random variables sampled from some distribution. Then one can compute statistics for the total roundoff error and estimate the probability that it is bounded by tu for a given t.…”