A Low-Cost Fault-Tolerant Approach for Hardware Implementation of Artificial Neural Networks

Ahmadi, Ahmad; Sargolzaie, M.H.; Fakhraie, S.M.; Lucas, Caro; Vakili, Shervin

doi:10.1109/iccet.2009.204

Cited by 15 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, [90] introduces a method of adding spare Pes, which can be reconfigured to replace any neuron. It can be used for both fault detection and correction, but it is limited to spatial architecture and can be used to recover from limited faulty PEs.…”

Section: ) Resilience Enhancement By Redundancymentioning

confidence: 99%

Dependable DNN Accelerator for Safety-Critical Systems: A Review on the Aging Perspective

Moghaddasi,

Gorgin,

Lee

2023

IEEE Access

View full text Add to dashboard Cite

Nowadays, artificial intelligence (AI) and deep learning (DL) progressively adapt to various spheres of our lives. These disciplines contain safety-critical applications such as autonomous driving with a high risk of human injury in the case of malfunction, requiring a high promise of dependability. Even the dependability becomes more crucial as shrinking CMOS technology feature size worsens the resilience concerns due to factors like aging. This paper addresses the overarching dependability issue of advanced deep neural networks (DNN) accelerators from the aging perspective. Especially, a comprehensive survey and taxonomy of techniques used to evaluate and mitigate aging effects are introduced. We cover different aging effects like permanent faults, timing errors, and lifetime issues. We review research by the layer-wise approach and categorize several resilience classes to bring out major features. The concluding part of this review highlights the questions answered and several future research directions. This study is expected to benefit researchers in different areas of DNN deployment, especially the dependability of this emergent paradigm.

show abstract

Section: ) Resilience Enhancement By Redundancymentioning

confidence: 99%

Dependable DNN Accelerator for Safety-Critical Systems: A Review on the Aging Perspective

Moghaddasi,

Gorgin,

Lee

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Ching-Tai Chiu et al [54] propose to add additional hidden nodes to avoid model accuracy loss and repeatedly remove nodes that do not significantly affect the network output. A. Ahmadi et al [55] proposed to add a spare neuron which can be reconfigured to compare with any neuron in the model. It can be used for both fault detection and correction, but it is limited to spatial neural network architecture and can only be used to recover from single faults.…”

Section: Model-layer Fault Tolerance a Related Workmentioning

confidence: 99%

Fault-Tolerant Deep Learning: A Hierarchical Perspective

Liu¹,

Gao²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the rapid advancements of deep learning in the past decade, it can be foreseen that deep learning will be continuously deployed in more and more safety-critical applications such as autonomous driving and robotics. In this context, reliability turns out to be critical to the deployment of deep learning in these applications and gradually becomes a firstclass citizen among the major design metrics like performance and energy efficiency. Nevertheless, the back-box deep learning models combined with the diverse underlying hardware faults make resilient deep learning extremely challenging. In this special session, we conduct a comprehensive survey of fault-tolerant deep learning design approaches with a hierarchical perspective and investigate these approaches from model layer, architecture layer, circuit layer, and cross layer respectively.

show abstract

“…For a fixed architecture, 2 we should minimize the generalization error with respect to λ. Since the MPE values,Ē(D f ) b andĒ(D f ) β , must be evaluated with the weight vector and the weight vector is also a function of λ, there is no a simple exact close form expression for the optimal value of λ.…”

Section: Selection Of Weight Decay Parametermentioning

confidence: 99%

“…However, the training speed is quite slow. The replication technique [2], [21], [37], [44] in which hidden nodes are replicated from a trained network, is an effective method to improve the fault tolerance but this approach needs to use additional sources. In [12], the equicontinuous properties of sigmoidal feedforward networks were analyzed.…”

mentioning

confidence: 99%

On the Selection of Weight Decay Parameter for Faulty Networks

Leung

Wang

Sum

2010

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

Abstract-The weight-decay technique is an effective approach to handle overfitting and weight fault. For fault-free networks, without an appropriate value of decay parameter, the trained network is either overfitted or underfitted. However, many existing results on the selection of decay parameter focus on fault-free networks only. It is well known that the weight-decay method can also suppress the effect of weight fault. For the faulty case, using a test set to select the decay parameter is not practice because there are huge number of possible faulty networks for a trained network. This paper develops two mean prediction error (MPE) formulae for predicting the performance of faulty radial basis function (RBF) networks. Two fault models, multiplicative weight noise and open weight fault, are considered. Our MPE formulae involve the training error and trained weights only. Besides, in our method, we do not need to generate a huge number of faulty networks to measure the test error for the fault situation. The MPE formulae allow us to select appropriate values of decay parameter for faulty networks. Our experiments showed that, although there are small differences between the true test errors (from the test set) and the MPE values, the MPE formulae can accurately locate the appropriate value of the decay parameter for minimizing the true test error of faulty networks.

show abstract

A Low-Cost Fault-Tolerant Approach for Hardware Implementation of Artificial Neural Networks

Cited by 15 publications

References 6 publications

Dependable DNN Accelerator for Safety-Critical Systems: A Review on the Aging Perspective

Dependable DNN Accelerator for Safety-Critical Systems: A Review on the Aging Perspective

Fault-Tolerant Deep Learning: A Hierarchical Perspective

On the Selection of Weight Decay Parameter for Faulty Networks

Contact Info

Product

Resources

About