Fault tolerance in neural networks: Neural design and hardware implementation

The Impact of Proton-Induced Single Events on Image Classification in a Neuromorphic Computing Architecture

“…In a similar manner to software neural networks, neuromorphic architectures are densely interconnected with large fan out to other neurons. Consequently, the architecture possesses increased redundancy compared to traditional computing architectures and can be fault tolerant if specifically designed that way, but comes at a computational cost [2][3][4][5]. The way neuromorphic computers "learn" is through the use of self-tuning weights.…”

“…As a result, neuromorphic architectures have increased redundancy and a reduction in the memory bandwidth bottleneck compared to traditional computing architectures [2]. While research is limited on how single events affect the relatively new architectures, studies have previously been conducted on realizations in COTs (Commercial off the Shelf), FPGAs (Field Programmable Gate Arrays), GPUs (Graphic Processing Units), and software frameworks [3,[5][6][7]. Several works in the literature analyze errors in neural networks through fault injection simulations [3][4][6][7][8][9], radiation tests [3,[6][7][8], and pulsed-laser tests [10].…”

“…Artificial neural networks are claimed to be intrinsically fault tolerant; however, most artificial neural networks cannot be considered fault tolerant without a proper design. Passive fault tolerance can be achieved through redundancy or by modifying learning [4][5]. Including faults during training promotes better generalization than other limited solutions, but it comes at both a higher computational cost and longer time for training [4][5].…”

“…Passive fault tolerance can be achieved through redundancy or by modifying learning [4][5]. Including faults during training promotes better generalization than other limited solutions, but it comes at both a higher computational cost and longer time for training [4][5]. Active fault tolerance such as resetting the neural network to a faultless state after a fault occurs and propagates is the most widely used design of fault tolerant neural networks in hardware [4][5].…”

On the Resilience of RTL NN Accelerators: Fault Characterization and Mitigation

“…These techniques can be potentially customized to mitigate faults of NNs, as well; however, with timing, area, or power costs. Also, techniques adapted for NNs are surveyed in [40], such as explicit redundancy, retraining, and modifying learning/inference phases. In this paper, instead of costly fault tolerance operations we present an application-aware fault mitigation in NNs, which does not require to exploit any redundant data bits or additional considerable overheads.…”