Impact of Tensor Cores and Mixed Precision on the Reliability of Matrix Multiplication in GPUs

Basso, Paula Cristina; Santos, Fernando Fernandes dos; Rech, Paolo

doi:10.1109/tns.2020.2977583

Cited by 23 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This happens regardless of the convolution type. Again, this is in contrast with data observed for GPUs, for which the magnitude of the error can be significantly higher (orders of magnitude) [11], [32]. This is another promising result for the TPU reliability in executing CNNs, as a higher error magnitude can have a greater impact on the output value.…”

Section: A Atomic Operationsmentioning

confidence: 65%

“…The fact that simple corrupted elements is the more common distribution for the TPU is in contrast with what has been observed for Graphics Processing Units (GPUs) [11], [31], [32], for which the majority of the corrupted matrices have multiple corrupted elements. This is due to the different way matrix multiplication is implemented.…”

Section: A Atomic Operationsmentioning

confidence: 75%

See 1 more Smart Citation

High Energy and Thermal Neutron Sensitivity of Google Tensor Processing Units

Rech

Malde

Cazzaniga

et al. 2022

IEEE Trans. Nucl. Sci.

Self Cite

View full text Add to dashboard Cite

In this paper we investigate the reliability of Google's Coral Tensor Processing Units (TPUs) to both high energy atmospheric neutrons (at ChipIR) and thermal neutrons from a pulsed source (at EMMA) and from a reactor (at TENIS). We report data obtained with an overall fluence of 3.41 × 10 12 n/cm 2 for atmospheric neutrons (equivalent to more than 30 million years of natural irradiation) and of 7.55×10 12 n/cm 2 for thermal neutrons. We evaluate the behavior of TPUs executing elementary operations with increasing input sizes (standard convolutions or depthwise convolutions) as well as eight CNNs configurations (SSD MobileNet v2 and SSD MobileDet, trained with COCO dataset, and Inception v4 and ResNet-50, with ILSVRC2012 dataset). We found that, despite the high error rate, most neutrons-induced errors only slightly modify the convolution output and do not change the CNNs detection or classification. By reporting details about the error model we provide valuable information on how to design the CNNs to avoid neutron-induced events to lead to miss detections or classifications.

show abstract

Section: A Atomic Operationsmentioning

confidence: 65%

Section: A Atomic Operationsmentioning

confidence: 75%

High Energy and Thermal Neutron Sensitivity of Google Tensor Processing Units

Rech

Malde

Cazzaniga

et al. 2022

IEEE Trans. Nucl. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…205 racy of such reduced or mixed-precision operations [14,18,5]. Several experiments are conducted later in this article to assess those effects on our convolution algorithm.…”

Section: Several Authors Have Explored the Arithmetic Accu-mentioning

confidence: 99%

Computing large 2D convolutions on GPU efficiently with the im2tensor algorithm

Seznec

Gac

Orieux

et al. 2022

J Real-Time Image Proc

View full text Add to dashboard Cite

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

show abstract

“…Another study by Basso et al (2020) explores the reliability of tensor cores in terms of rate of hardware errors in matrix multiplications. The main finding is that low-precision operations and usage of tensor cores increase the amount of correct data produced by the GPU, despite increasing the impact of numerical errors due to the use of lower-precision data.…”

Section: Previous Workmentioning

confidence: 99%

Peer Review #2 of "Numerical behavior of NVIDIA tensor cores (v0.2)"

2021

View full text Add to dashboard Cite

We explore the floating-point arithmetic implemented in the NVIDIA tensor cores, which are hardware accelerators for mixed-precision matrix multiplication available on the Volta, Turing, and Ampere microarchitectures. Using Volta V100, Turing T4 and Ampere A100 graphics cards, we determine what precision is used for the intermediate results, whether subnormal numbers are supported, what rounding mode is used, in which order the operations underlying the matrix multiplication are performed, and whether partial sums are normalized. These aspects are not documented by NVIDIA, and we gain insight by running carefully designed numerical experiments on these hardware units. Knowing the answers to these questions is important if one wishes to: 1) accurately simulate NVIDIA tensor cores on conventional hardware; 2) understand the differences between results produced by code that utilizes tensor cores and code that uses only IEEE 754-compliant arithmetic operations; and 3) build custom hardware whose behavior matches that of NVIDIA tensor cores. As part of this work we provide a test suite that can be easily adapted to test newer versions of the NVIDIA tensor cores as well as similar accelerators from other vendors, as they become available. Moreover, we identify a non-monotonicity issue affecting floating point multi-operand adders if the intermediate results are not normalized after each step.

show abstract

Impact of Tensor Cores and Mixed Precision on the Reliability of Matrix Multiplication in GPUs

Cited by 23 publications

References 10 publications

High Energy and Thermal Neutron Sensitivity of Google Tensor Processing Units

High Energy and Thermal Neutron Sensitivity of Google Tensor Processing Units

Computing large 2D convolutions on GPU efficiently with the im2tensor algorithm

Peer Review #2 of "Numerical behavior of NVIDIA tensor cores (v0.2)"

Contact Info

Product

Resources

About