When Single Event Upset Meets Deep Neural Networks: Observations, Explorations, and Remedies

Yan, Zheyu; Shi, Yiyu; Liao, Wang; Hashimoto, Masanori; Zhou, Xichuan; Zhuo, Cheng

doi:10.1109/asp-dac47756.2020.9045134

Cited by 36 publications

(14 citation statements)

References 27 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The data format itself obviously decides or affects the data range. [52] found out that the errors in exponent bits of the 32bit floatingpoint weights have large impacts on the performance. [23] investigated the resilience characteristics of several floatingpoint and non-dynamic fixed-point representations.…”

Section: Discussionmentioning

confidence: 99%

FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture

Ning¹,

Ge²,

Li³

et al. 2020

Preprint

View full text Add to dashboard Cite

With the fast evolvement of embedded deep-learning computing systems, applications powered by deep learning are moving from the cloud to the edge. When deploying neural networks (NNs) onto the devices under complex environments, there are various types of possible faults: soft errors caused by cosmic radiation and radioactive impurities, voltage instability, aging, temperature variations, and malicious attackers. Thus the safety risk of deploying NNs is now drawing much attention. In this paper, after the analysis of the possible faults in various types of NN accelerators, we formalize and implement various fault models from the algorithmic perspective. We propose Fault-Tolerant Neural Architecture Search (FT-NAS) to automatically discover convolutional neural network (CNN) architectures that are reliable to various faults in nowadays devices. Then we incorporate fault-tolerant training (FTT) in the search process to achieve better results, which is referred to as FTT-NAS. Experiments on CIFAR-10 show that the discovered architectures outperform other manually designed baseline architectures significantly, with comparable or fewer floatingpoint operations (FLOPs) and parameters. Specifically, with the same fault settings, F-FTT-Net discovered under the feature fault model achieves an accuracy of 86.2% (VS. 68.1% achieved by MobileNet-V2), and W-FTT-Net discovered under the weight fault model achieves an accuracy of 69.6% (VS. 60.8% achieved by . By inspecting the discovered architectures, we find that the operation primitives, the weight quantization range, the capacity of the model, and the connection pattern have influences on the fault resilience capability of NN models.

show abstract

Section: Discussionmentioning

confidence: 99%

FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture

Ning¹,

Ge²,

Li³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Fault-masking techniques are conceptually simple. We could use error-correcting codes (ECC) to protect memory elements [35] and triple-modular redundancy (TMR) to protect computational units [29,42]. However, the corresponding hardware overhead is exceptionally high.…”

Section: Limitations and Future Workmentioning

confidence: 99%

DeepDyve: Dynamic Verification for Deep Neural Networks

Min

Luo

et al. 2020

Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security

View full text Add to dashboard Cite

Deep neural networks (DNNs) have become one of the enabling technologies in many safety-critical applications, e.g., autonomous driving and medical image analysis. DNN systems, however, suffer from various kinds of threats, such as adversarial example attacks and fault injection attacks. While there are many defense methods proposed against maliciously crafted inputs, solutions against faults presented in the DNN system itself (e.g., parameters and calculations) are far less explored. In this paper, we develop a novel lightweight fault-tolerant solution for DNN-based systems, namely DeepDyve, which employs pre-trained neural networks that are far simpler and smaller than the original DNN for dynamic verification. The key to enabling such lightweight checking is that the smaller neural network only needs to produce approximate results for the initial task without sacrificing fault coverage much. We develop efficient and effective architecture and task exploration techniques to achieve optimized risk/overhead trade-off in Deep-Dyve. Experimental results show that DeepDyve can reduce 90% of the risks at around 10% overhead. CCS CONCEPTS• Computing methodologies → Neural networks; • Hardware → System-level fault tolerance.

show abstract

“…Different strategies have been proposed to tackle these issues. Noise-aware training [4] and uncertainty-aware neural architecture search [10][11][12] aim at fortifying DNNs so that their performance remains mostly unaffected even in the presence of device variations. However, these methods are not economical because they require re-training DNNs from scratch and cannot make use of existing pre-trained models.…”

Section: Introductionsmentioning

confidence: 99%

SWIM: Selective Write-Verify for Computing-in-Memory Neural Accelerators

Yan¹,

Hu²,

Shi³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Computing-in-Memory architectures based on non-volatile emerging memories have demonstrated great potential for deep neural network (DNN) acceleration thanks to their high energy efficiency. However, these emerging devices can suffer from significant variations during the mapping process (i.e., programming weights to the devices), and if left undealt with, can cause significant accuracy degradation. The non-ideality of weight mapping can be compensated by iterative programming with a write-verify scheme, i.e.,

show abstract

When Single Event Upset Meets Deep Neural Networks: Observations, Explorations, and Remedies

Cited by 36 publications

References 27 publications

FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture

FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture

DeepDyve: Dynamic Verification for Deep Neural Networks

SWIM: Selective Write-Verify for Computing-in-Memory Neural Accelerators

Contact Info

Product

Resources

About