Rescuing Memristor-based Neuromorphic Design with High Defects

Abstract

Memristor-based synaptic network has been widely investigated and applied to neuromorphic computing systems for the fast computation and low design cost. As memristors continue to mature and achieve higher density, bit failures within crossbar arrays can become a critical issue. These can degrade the computation accuracy significantly. In this work, we propose a defect rescuing design to restore the computation accuracy. In our proposed design, significant weights in a specified network are first identified and retraining and remapping algorithms are described. For a two layer neural network with 92.64% classification accuracy on MNIST digit recognition, our evaluation based on real device testing shows that our design can recover almost its full performance when 20% random defects are present.

The future of electronics based on memristive systems

“…Other work 47 has shown that selectively optimizing the operating point for ON versus OFF switching, which can involve modifying the applied voltages and pulse widths, can produce acceptable results, but that the percentage of fully stuck ON or OFF devices can, in turn, lead to large errors. Nonetheless, many studies have shown that by retraining the network, in the presence of stuck ON or OFF cells, can almost fully compensate for the defects and regain the classification accuracy, even for up to 20% defects 48 . Some researchers have also studied offline trained systems, focusing on developing inference-only neural network accelerators 39,49 that can outperform the current state-of-the-art systems based on graphics processing units.…”

A Survey of ReRAM-Based Architectures for Processing-In-Memory and Neural Networks

“…Reliability Challenges of ReRAM: The high defect rate and PV leads to reliability issues [21,37]. For example, due to "single-bit failure", a cell may get stuck at high or low conductance value, called stuck-at-one or stuck-at-zero (SA1/SA0), respectively.…”

“…Storing positive and negative weights in different MCAs [12,18,21,38,43,74] Using only non-negative weights [50] Using multiple ReRAMs to overcome the precision limitation of ReRAM [17,18] Using tiled designs to avoid using large MCAs and/or to achieve fine-grain reconfigurability [15][16][17]41,46,56,57,70] Mapping largest weights to variation/fault-free MCAs to minimize errors [69] Assigning larger weights to MSB and smaller weights to LSB [58] Distinguishing between critical and non-critical weights [37] Avoiding costly SET operations in ReRAM [24] Hu et al [73] developed an algorithm for transforming arbitrary matrix values into memristor conductances for minimizing inaccuracies in MVM while accounting for memristor crossbar array (MCA) circuit limitations. Figure 3 shows the overall flow of their technique.…”

“…Their fault-tolerance technique improves accuracy significantly for both low-endurance cells and cells with high endurance but a large fraction of initial faults. Liu et al [37] presented a technique for tolerating stuck-at-faults in ReRAM-based NCSs. Figure 22 shows the overall design flow of their technique.…”

Towards accurate and high-speed spiking neuromorphic systems with data quantization-aware deep networks

“…Similarly, the synaptic weights in the memristor-based designs are usually represented by three or four bits data. Although the memristor devices can afford continuous conductance states or 6-bit (64 levels) as was reported by HP Labs [16], the heavy programming cost in speed and circuit design are not acceptable. In these SNC designs, the neuron signals are rate coded and the signal strength is represented by spike numbers in a time window in discrete values.…”