Design of CMOS-memristor Circuits for LSTM architecture

IEEE Trans. Neural Netw. Learning Syst.

James

Chua

2020

Self Cite

218

120

The volume, veracity, variability and velocity of data produced from the ever increasing network of sensors connected to Internet pose challenges for power management, scalability and sustainability of cloud computing infrastructure. Increasing the data processing capability of edge computing devices at lower power requirements can reduce several overheads for cloud computing solutions. This paper provides the review of neuromorphic CMOS-memristive architectures that can be integrated into edge computing devices. We discuss why the neuromorphic architectures are useful for edge devices and show the advantages, drawbacks and open problems in the field of neuro-memristive circuits for edge computing.

Section: )mentioning

confidence: 99%

Neuromemristive Circuits for Edge Computing: A Review

IEEE Trans. Neural Netw. Learning Syst.

James

Chua

2020

Self Cite

218

120

“…LSTM is used in a wide range of applications in the contextual data processing based on prediction making and natural language processing. Hardware implementation of LSTM is a new topic studied in [43], [44].…”

Section: Background a Learning Algorithms And Biologically Inspimentioning

confidence: 99%

“…The full implementation of LSTM with analog circuits has not been proposed yet. However, the analog implementation of the separate LSTM components has been shown in [43], [44].…”

Section: Long Short Term Memorymentioning

confidence: 99%

Learning in Memristive Neural Network Architectures Using Analog Backpropagation Circuits

IEEE Trans. Circuits Syst. I

Salama

James

2019

Self Cite

121

The on-chip implementation of learning algorithms would speed-up the training of neural networks in crossbar arrays. The circuit level design and implementation of backpropagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we proposed the analog backpropagation learning circuits for various memristive learning architectures, such as Deep Neural Network (DNN), Binary Neural Network (BNN), Multiple Neural Network (MNN), Hierarchical Temporal Memory (HTM) and Long-Short Term Memory (LSTM). The circuit design and verification is done using TSMC 180nm CMOS process models, and TiO2 based memristor models. The application level validations of the system are done using XOR problem, MNIST character and Yale face image databases.

“…Fig. 1 (b) illustrates the memristive hardware implementation of gated LSTM proposed in [6], [11]. In this work, classification using LSTM was performed using two layer network topology: LSTM layer with LSTM units unrolled for 151 time-steps and ANN layer with linear activation function.…”

Section: Quality Inspectionmentioning

confidence: 99%

Wafer Quality Inspection using Memristive LSTM, ANN, DNN and HTM

Adam

Smagulova

2018 IEEE Electrical Design of Advanced Packaging and Systems Symposium (EDAPS)

et al. 2018

Self Cite

The automated wafer inspection and quality control is complex and time consuming task, which can be speed up using neuromorphic memristive architectures, as a separate inspection device or integrating directly into sensors. This paper presents the performance analysis and comparison of different neuromorphic architectures for patterned wafer quality inspection and classification. The application of non-volatile memristive devices in these architectures ensures low power consumption, small on-chip area scalability. We demonstrate that Long-Short Term Memory (LSTM) outperforms other architectures for the same number of training iterations, and has relatively low on-chip area and power consumption.