SkyNet: Memristor-based 3D IC for artificial neural networks

Bhat, Sachin; Kulkami, Sourabh; Shi, Jiajun; Li, Mingyu; Moritz, Csaba Andras

doi:10.1109/nanoarch.2017.8053706

Cited by 9 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In-memory computing with the memristor crossbar in figure 3(c) can be used to overcome the von Neumann machine's memory access bottleneck mentioned earlier. Memristors are nonvolatile memories that allow for fast and energy-efficient read and write operations and they can be stacked layer by layer for forming 3D structure (Strukov et al 2008, Jo et al 2010, Truong et al 2014, Hu et al 2014, Adam et al 2016, Bhat et al 2017, Chakrabarti et al 2017, Li et al 2018b, Li and Belkin 2018, Li et al 2018a, Jeong and Shi 2018, Sheng et al 2019, James 2019, Amirsoleimani et al 2020, Lin et al 2020, Wang et al 2020. Memristor fabrication can be combined with conventional CMOS processing technology, where memristor crossbars can be integrated with CMOS devices.…”

Section: Introductionmentioning

confidence: 99%

Quantization, training, parasitic resistance correction, and programming techniques of memristor-crossbar neural networks for edge intelligence

Nguyen

et al. 2022

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

In Internet-of-Things (IoT) era, edge intelligence is critical for overcoming the communication and computing energy crisis, which is unavoidable if cloud computing is used exclusively. Memristor crossbars with in-memory computing may be suitable for realizing edge intelligence hardware. They can perform both memory and computing functions, allowing for the development of low-power computing architectures that go beyond the Von Neumann computer. For implementing edge-intelligence hardware with memristor crossbars, in this paper, we review various techniques such as quantization, training, parasitic resistance correction, and low-power crossbar programming, and so on. In particular, memristor crossbars can be considered to realize quantized neural networks with binary and ternary synapses. For preventing memristor defects from degrading edge intelligence performance, chip-in-the-loop training can be useful when training memristor crossbars. Another undesirable effect in memristor crossbars is parasitic resistances such as source, line, and neuron resistance, which worsens as crossbar size increases. Various circuit and software techniques can compensate for parasitic resistances like source, line, and neuron resistance. Finally, we discuss an energy-efficient programming method for updating synaptic weights in memristor crossbars, which is needed for learning the edge devices.

show abstract

Section: Introductionmentioning

confidence: 99%

Quantization, training, parasitic resistance correction, and programming techniques of memristor-crossbar neural networks for edge intelligence

Nguyen

et al. 2022

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

show abstract

“…The memristive networks are inspired from the biological concepts of human brain processing that has been developed to replace the conventional Von Neumann computing architecture in the future [ 5 , 6 , 7 , 8 ]. For implementing the non-Von-Neumann computing architecture, memristors can provide various advantages of scalability, low-energy consumption, non-volatility, and potential 3-dimensional stacking [ 9 , 10 , 11 ] since its first experimental demonstration [ 12 ]. Figure 1 a shows the conceptual diagram of the cloud systems, edge-computing devices, and Internet-of-Things (IoT) sensors [ 13 , 14 , 15 ].…”

Section: Introductionmentioning

confidence: 99%

Asymmetrical Training Scheme of Binary-Memristor-Crossbar-Based Neural Networks for Energy-Efficient Edge-Computing Nanoscale Systems

2019

View full text Add to dashboard Cite

For realizing neural networks with binary memristor crossbars, memristors should be programmed by high-resistance state (HRS) and low-resistance state (LRS), according to the training algorithms like backpropagation. Unfortunately, it takes a very long time and consumes a large amount of power in training the memristor crossbar, because the program-verify scheme of memristor-programming is based on the incremental programming pulses, where many programming and verifying pulses are repeated until the target conductance. Thus, this reduces the programming time and power is very essential for energy-efficient and fast training of memristor networks. In this paper, we compared four different programming schemes, which are F-F, C-F, F-C, and C-C, respectively. C-C means both HRS and LRS are coarse-programmed. C-F has the coarse-programmed HRS and fine LRS, respectively. F-C is vice versa of C-F. In F-F, both HRS and LRS are fine-programmed. Comparing the error-energy products among the four schemes, C-F shows the minimum error with the minimum energy consumption. The asymmetrical coarse HRS and fine LRS can reduce the time and energy during the crossbar training significantly, because only LRS is fine-programmed. Moreover, the asymmetrical C-F can maintain the network’s error as small as F-F, which is due to the coarse-programmed HRS that slightly degrades the error.

show abstract

“…Since then, they have been intensively studied as a possible candidate for implementing neural-networks in nanoscale [2]. Memristor crossbars can be built in three-dimensional architecture, which seems to be very similar to the biological neuronal structure that was observed in mammalian brains [3,4,5]. Moreover, memristor crossbars can be fabricated while using the Back-End-Of-Line process on the top of Silicon substrate [3,4].…”

Section: Introductionmentioning

confidence: 99%

“…Memristor crossbars can be built in three-dimensional architecture, which seems to be very similar to the biological neuronal structure that was observed in mammalian brains [3,4,5]. Moreover, memristor crossbars can be fabricated while using the Back-End-Of-Line process on the top of Silicon substrate [3,4]. Additionally, their non-volatile and non-linear behaviors can be useful in performing cognitive computing with memristor crossbars [6,7].…”

Section: Introductionmentioning

confidence: 99%

Partial-Gated Memristor Crossbar for Fast and Power-Efficient Defect-Tolerant Training

2019

View full text Add to dashboard Cite

A real memristor crossbar has defects, which should be considered during the retraining time after the pre-training of the crossbar. For retraining the crossbar with defects, memristors should be updated with the weights that are calculated by the back-propagation algorithm. Unfortunately, programming the memristors takes a very long time and consumes a large amount of power, because of the incremental behavior of memristor’s program-verify scheme for the fine-tuning of memristor’s conductance. To reduce the programming time and power, the partial gating scheme is proposed here to realize the partial training, where only some part of neurons are trained, which are more responsible in the recognition error. By retraining the part, rather than the entire crossbar, the programming time and power of memristor crossbar can be significantly reduced. The proposed scheme has been verified by CADENCE circuit simulation with the real memristor’s Verilog-A model. When compared to retraining the entire crossbar, the loss of recognition rate of the partial gating scheme has been estimated only as small as 2.5% and 2.9%, for the MNIST and CIFAR-10 datasets, respectively. However, the programming time and power can be saved by 86% and 89.5% than the 100% retraining, respectively.

show abstract

SkyNet: Memristor-based 3D IC for artificial neural networks

Cited by 9 publications

References 18 publications

Quantization, training, parasitic resistance correction, and programming techniques of memristor-crossbar neural networks for edge intelligence

Quantization, training, parasitic resistance correction, and programming techniques of memristor-crossbar neural networks for edge intelligence

Asymmetrical Training Scheme of Binary-Memristor-Crossbar-Based Neural Networks for Energy-Efficient Edge-Computing Nanoscale Systems

Partial-Gated Memristor Crossbar for Fast and Power-Efficient Defect-Tolerant Training

Contact Info

Product

Resources

About