Can We Avoid Rounding-Error Estimation in HPC Codes and Still Get Trustworthy Results?

Abstract

Numerical validation enables one to ensure the reliability of numerical computations that rely on floating-point operations. Discrete Stochastic Arithmetic (DSA) makes it possible to validate the accuracy of floating-point computations using random rounding. However, it may bring a large performance overhead compared with the standard floatingpoint operations. In this article, we show that with perturbed data it is possible to use standard floating-point arithmetic instead of DSA for the purpose of numerical validation. For instance, for codes including matrix multiplications, we can directly utilize the matrix multiplication routine (GEMM) of level-3 BLAS that is performed with standard floating-point arithmetic. Consequently, we can achieve a significant performance improvement by avoiding the performance overhead of DSA operations as well as by exploiting the speed of highly-optimized BLAS implementations. Finally, we demonstrate the performance gain using Intel MKL routines compared against the DSA version of BLAS routines.

Parallel and accurate k‐means algorithm on CPU‐GPU architectures for spectral clustering

“…In the Update routine, the algorithm needs to calculate the sum of data instances in each cluster and then divide the sum by the number of instances in the cluster. Therefore, when a large number of instances are added together one by one naively, the accumulation of rounding errors that may occur finally impairs the clustering quality (see Reference 24 for more illustration of the effect of rounding errors). On the other hand, using double precision ( 64‐bits arithmetic ) can reduce the effect of rounding errors to a satisfying level of accuracy in our use case, but the computational cost is higher (see, e.g., Reference 25).…”

“…Then the nearest centroid for each instance can be found and recorded (lines [16][17][18]. Finally the cluster label of each instance is updated according to its nearest centroid, and the changes of label are counted into the private track of each thread (lines [22][23][24][25]. The reduction directive sums the private track of all threads (line 3).…”

“…For the coalescence of memory access, we use the transposed matrix of data instances (line 18). Then the nearest centroid for each instance can be found and recorded (lines[22][23][24][25]. The cluster label will be changed if necessary, and the change will be marked with 1 in the shared 1D block array shTrack (lines[28][29][30][31].__global__ void ComputeAssign (T_real *GPU_dataT, T_real *GPU_cent, int *GPU_label, unsigned long long int *AdrGPU_track_sum) { int idx = blockIdx.x * BSXN + threadIdx.x; __shared__ unsigned long long int shTrack[BSXN]; shTrack[threadIdx.x] = 0; if (idx < n) { int min = 0; T_real diff, dist_sq, minDist_sq; for (int k = 0; k < kc; k++) { dist_sq = 0.0f; // Calculate the square of distance // between instance idx and centroid k // across nd dimensions for (int j = 0; j < nd; j++) { diff = GPU_dataT[j*n + idx] -GPU_cent[k*nd + j]; dist_sq += (diff*diff); } // Find the nearest centroid to instance idx if (dist_sq < minDist_sq || k == 0the changes of label into "track": // two-part reduction // 1 -Parallel reduction of 1D block shared array ... // shTrack[*] into shTrack[0], ... // kill useless threads step by step, ... // only thread 0 survives at the end // 2 -Final reduction into a global array if (shTrack[0] > 0) atomicAdd(AdrGPU_track_sum, shTrack[0]); } ComputeAssign routine on CPU Finally, we count the changes of label by a two-part reduction.…”

FPGA Implementation of Embedded Floating-Point Core with Microarchitectural Support

“…Generally, FMA achieves matrix multiplication which is widely used in Convolutional Neural Networks (CNN) [19] and basic linear algebra subprograms (BLAS) [21,22]. Hardware implementations for matrix multiplication have problems with the requirement of many hardware resources [19,20], software interferences [21], memory requirements, and numerical inefficacy [23,24]. Large-size hardware shifters and CSA are required for conventional FMA architectures.…”