Cases Study of Inputs Split Based Calibration Method for RRAM Crossbar

Sun, Siqi; Xu, Hui; Li, Jiwei; Li, Zhiwei; Sun, Yi; Li, Qingjiang; Liu, Haijun

doi:10.1109/access.2019.2944417

Cited by 2 publications

(3 citation statements)

References 37 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remap HW redundancy [7] HW overhead (crossbar) Neuron permutation [21], [23] Inapplicable to conv. layer Row permutation [17], [24] HW overhead (router) Row flipping [23] HW overhead (negation) Input splitting [26] HW overhead (input split)…”

Section: A Saf Mitigation Techniquesmentioning

confidence: 99%

“…Our method, FPT, does not require any training or training dataset if there is a pre-trained model, whereas the method in [29] would still require additional training with a training dataset even if a pre-trained model is given. Lastly, [26] is a calibration method similar to ours. However, it relies on input preprocessing (called input splitting), which must be performed at runtime and therefore requires extra hardware unlike ours.…”

Section: A Saf Mitigation Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Training-Free Stuck-At Fault Mitigation for ReRAM-Based Deep Learning Accelerators

Quan

Fouda

Lee

et al. 2023

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Although Resistive RAMs can support highly efficient matrix-vector multiplication, which is very useful for machine learning and other applications, the non-ideal behavior of hardware such as stuck-at fault and IR drop is an important concern in making ReRAM crossbar array-based deep learning accelerators. Previous work has addressed the nonideality problem through either redundancy in hardware, which requires a permanent increase of hardware cost, or software retraining, which may be even more costly or unacceptable due to its need for a training dataset as well as high computation overhead. In this paper we propose a very light-weight method that can be applied on top of existing hardware or software solutions. Our method, called FPT (Forward-Parameter Tuning), takes advantage of a certain statistical property existing in the activation data of neural network layers, and can mitigate the impact of mild nonidealities in ReRAM crossbar arrays for deep learning applications without using any hardware, a dataset, or gradientbased training. Our experimental results using MNIST, CIFAR-10, CIFAR-100, and ImageNet datasets in binary and multibit networks demonstrate that our technique is very effective, both alone and together with previous methods, up to 20% fault rate, which is higher than even some of the previous remapping methods. We also evaluate our method in the presence of other nonidealities such as variability and IR drop. Further, we provide an analysis based on the concept of effective fault rate, which not only demonstrates that effective fault rate can be a useful tool to predict the accuracy of faulty RCA-based neural networks, but also explains why mitigating the SAF problem is more difficult with multi-bit neural networks.

show abstract

Section: A Saf Mitigation Techniquesmentioning

confidence: 99%

Section: A Saf Mitigation Techniquesmentioning

confidence: 99%

Training-Free Stuck-At Fault Mitigation for ReRAM-Based Deep Learning Accelerators

Quan

Fouda

Lee

et al. 2023

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

show abstract

“…After that, when the retraining algorithm fails to improve the accuracy, redundant memristor columns are used for remapping the algorithm [25]. In order to keep up the application performance running on the memristor array, Sun et al propose a software-based calibration method, which splits the input vector into two parts, standard vector and another input [26]. Apart from that, a method using binary memristor crossbars constructs ternary CNN is proposed by them, and a training strategy which considers the position of a damaged device in the crossbar during training is utilized to hold on to high neural network accuracy under low-yield conditions.…”

Section: Introductionmentioning

confidence: 99%

A training strategy for improving the robustness of memristor-based binarized convolutional neural networks

Huang¹,

Yu²,

Chen³

et al. 2021

Semicond. Sci. Technol.

Self Cite

View full text Add to dashboard Cite

Memristor-based convolutional neural networks (CNNs) have been extensively studied in the field of edge computing, owing to the efficient characteristics of memristors, such as high integration density and powerful processing capability. However, constrained by the low yield of memristor array and the memristance variation, memristor-based CNNs have failed to be widely applied. Consequently, a training strategy is proposed to improve the robustness of memristor-based binarized neural networks for prompting embedded application. Simulation results on the MNIST dataset reveal that this strategy is able to improve the performance of a memristor-based two-layer CNN with device defects. Specifically, when the yield rate of the memristor array is 60%, the recognition rate of a two-layer memristor-based binarized convolutional neural network achieves around 91.19%, and when the characteristic of device variation is 28%, it reaches about 91.53%.

show abstract

Cases Study of Inputs Split Based Calibration Method for RRAM Crossbar

Cited by 2 publications

References 37 publications

Training-Free Stuck-At Fault Mitigation for ReRAM-Based Deep Learning Accelerators

Training-Free Stuck-At Fault Mitigation for ReRAM-Based Deep Learning Accelerators

A training strategy for improving the robustness of memristor-based binarized convolutional neural networks

Contact Info

Product

Resources

About