An Efficient Fault-Tolerant Winograd Convolution for Convolutional Neural Networks

Kao, Yu-Hsin; Liu, Chia-Chun; Ding, Yiwen; Huang, Tsung-Chu

doi:10.1109/iet-iceta56553.2022.9971628

Cited by 1 publication

(1 citation statement)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although conventional fault-tolerant design approaches such as triple modular redundancy, dual modular redundancy (DMR), ECC, and algorithm-based fault tolerance (ABFT) [52]- [55] can potentially alleviate the influence of soft errors, they usually require considerable computing overhead. For instance, the authors in [56] have an self-checking scheme integrated in each modular of a typical DMR approach such that each modular can detect errors by itself. In this case, the revised DMR can also recover when only one of the modular is faulty.…”

Section: Fault Tolerance Of Neural Networkmentioning

confidence: 99%

Exploring Winograd Convolution for Cost-Effective Neural Network Fault Tolerance

Xue,

Liu,

Liu

et al. 2023

IEEE Trans. VLSI Syst.

View full text Add to dashboard Cite

Winograd is generally utilized to optimize convolution performance and computational efficiency because of the reduced multiplication operations, but the reliability issues brought by winograd are usually overlooked. In this work, we observe the great potential of winograd convolution in improving neural network (NN) fault tolerance. Based on the observation, we evaluate winograd convolution fault tolerance comprehensively from different granularities ranging from models, layers, and operation types for the first time. Then, we explore the use of inherent fault tolerance of winograd convolution for cost-effective NN protection against soft errors. Specifically, we mainly investigate how winograd convolution can be effectively incorporated with classical fault-tolerant design approaches including triple modular redundancy (TMR), faultaware retraining, and constrained activation functions. According to our experiments, winograd convolution can reduce the faulttolerant design overhead by 55.77% on average without any accuracy loss compared to standard convolution, and further reduce the computing overhead by 17.24% when the inherent fault tolerance of winograd convolution is considered. When it is applied on fault-tolerant neural networks enhanced with faultaware retraining and constrained activation functions, the resulting model accuracy generally shows significant improvement in presence of various faults.

show abstract

Section: Fault Tolerance Of Neural Networkmentioning

confidence: 99%