Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars

Bhattacharjee, Abhiroop; Kim, Youngeun; Moitra, Abhishek; Panda, Priyadarshini

doi:10.1145/3531437.3539729

Cited by 12 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, SNNs appear to be energy-efficient alternatives to ANNs, due to their brain-like computations and communications using sparse 1-bit spiking activations [13]. Besides, the robustness of ANNs and SNNs to noisy synaptic weights have been compared [12], [14]. However, authors in [12] do not consider a realistic hardware model, as weights are simulated with 32-bit floating point precision and only one type of error is considered.…”

Section: Introductionmentioning

confidence: 99%

“…However, authors in [12] do not consider a realistic hardware model, as weights are simulated with 32-bit floating point precision and only one type of error is considered. A more realistic hardware model of a RRAM crossbar implementation for evaluating the robustness of SNNs and ANNs is presented in an other work [14]. Nevertheless, none of these works [11], [12], [14] consider the benefits of injecting noise during training, which has proven very effective to enhance the fault tolerance of neural networks [15].…”

Section: Introductionmentioning

confidence: 99%

“…A more realistic hardware model of a RRAM crossbar implementation for evaluating the robustness of SNNs and ANNs is presented in an other work [14]. Nevertheless, none of these works [11], [12], [14] consider the benefits of injecting noise during training, which has proven very effective to enhance the fault tolerance of neural networks [15]. Strategies based on injecting noise during training have been demonstrated with NVM implementations [5], [6], [16], [17].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Dampfhoffer

Lopez

Mesquida

et al. 2023

2023 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

The implementation of Artificial Neural Networks (ANNs) using analog Non-Volatile Memories (NVMs) for synaptic weights storage promises improved energy-efficiency and higher density compared to fully-digital implementations. However, NVMs are prone to variability, resulting in a degradation of the accuracy of ANNs. In this paper, a general methodology to evaluate and enhance the accuracy of neural networks implemented with non-ideal multi-level NVMs is presented. A hardware fault model distinguishing two types of errors, namely static and dynamic, capturing the variability of NVMs is proposed. Considering various neural networks, it is shown that error-aware training highly increases the robustness to errors compared to a standard, error-agnostic, training. Moreover, Recurrent NNs (RNNs) and Spiking NNs (SNNs) are found to be inherently more robust to dynamic errors than Convolutional NNs (CNNs). In addition, new insights on the adaptability of neural networks to noisy multi-level NVMs are presented, which could further improve their robustness in this context. The methodology aims at providing tools for hardware-software co-design, paving the way for a broader use of multi-level NVM-based synapses.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Dampfhoffer

Lopez

Mesquida

et al. 2023

2023 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…[3]. Motivated by this, the robustness analysis and adversarial defense in AccSNNs have been thoroughly investigated in several recent works [4]- [7]. Very recently, in [8], the authors showed that approximate DNNs (AxDNNs) are more prone to adversarial attacks as compared to accurate DNNs (AccDNN).…”

Section: Introductionmentioning

confidence: 99%

Security-Aware Approximate Spiking Neural Networks

Ahmad¹,

Siddique²,

Hoque³

2023

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) and Spiking Neural Networks (SNNs) are both known for their susceptibility to adversarial attacks. Therefore, researchers in the recent past have extensively studied the robustness and defense of DNNs and SNNs under adversarial attacks. Compared to accurate SNNs (AccSNN), approximate SNNs (AxSNNs) are known to be up to 4X more energy-efficient for ultra-low power applications. Unfortunately, the robustness of AxSNNs under adversarial attacks is yet unexplored. In this paper, we first extensively analyze the robustness of AxSNNs with different structural parameters and approximation levels under two gradient-based and two neuromorphic attacks. Then, we propose two novel defense methods, i.e., precision scaling and approximate quantization-aware filtering (AQF), for securing AxSNNs. We evaluated the effectiveness of these two defense methods using both static and neuromorphic datasets. Our results demonstrate that AxSNNs are more prone to adversarial attacks than AccSNNs, but precision scaling and AQF significantly improve the robustness of AxSNNs. For instance, a PGD attack on AxSNN results in a 72% accuracy loss compared to AccSNN without any attack, whereas the same attack on the precision-scaled AxSNN leads to only a 17% accuracy loss in the static MNIST dataset (4X robustness improvement). Similarly, a Sparse Attack on AxSNN leads to a 77% accuracy loss when compared to AccSNN without any attack, whereas the same attack on an AxSNN with AQF leads to only a 2% accuracy loss in the neuromorphic DVS128 Gesture dataset (38X robustness improvement).

show abstract

“…A few works have investigated the impact of 'benign' nonideal properties in RRAM-based hardware on adversary attacks toward image classification. One work discusses how the nonideal properties reduce the adversary attack success rate and concludes that the RRAM-based neuromorphic hardware is inherently robust against adversary attacks (Bhattacharjee and Panda, 2020). While a recent work points out that the hard faults in the RRAM crossbar array can be leveraged to substantially enhance the adversary attack strength and effectively breakthrough software defense strategy (Lv et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Fault-Aware Adversary Attack Analyses and Enhancement for RRAM-Based Neuromorphic Accelerator

Shang

Jung

et al. 2022

Front. Sens.

View full text Add to dashboard Cite

Neural networks have been widely deployed in sensor networks and IoT systems due to the advance in lightweight design and edge computing as well as emerging energy-efficient neuromorphic accelerators. However, adversary attack has raised a major threat against neural networks, which can be further enhanced by leveraging the natural hard faults in the neuromorphic accelerator that is based on resistive random access memory (RRAM). In this paper, we perform a comprehensive fault-aware attack analysis method for RRAM-based accelerators by considering five attack models based on a wide range of device- and circuit-level nonideal properties. The research on nonideal properties takes into account detailed hardware situations and provides a more accurate perspective on security. Compared to the existing adversary attack strategy that only leverages the natural fault, we propose an initiative attack based on two soft fault injection methods, which do not require a high-precision laboratory environment. In addition, an optimized fault-aware adversary algorithm is also proposed to enhance the attack effectiveness. The simulation results of an MNIST dataset on a classic convolutional neural network have shown that the proposed fault-aware adversary attack models and algorithms achieve a significant improvement in the attacking image classification.

show abstract

Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars

Cited by 12 publications

References 14 publications

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

Security-Aware Approximate Spiking Neural Networks

Fault-Aware Adversary Attack Analyses and Enhancement for RRAM-Based Neuromorphic Accelerator

Contact Info

Product

Resources

About