Analog Weight Updates with Compliance Current Modulation of Binary ReRAMs for On-Chip Learning

Payvand, Melika; Demirağ, Yiğit; Dalgaty, Thomas; Vianello, Elisa; Indiveri, Giacomo

doi:10.1109/iscas45731.2020.9180808

Cited by 21 publications

(13 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To train the network, we utilize a technologically plausible training algorithm, namely a stochastic Delta rule algorithm, which is the simplest form of gradient descent for single-layer networks ( Payvand et al, 2020 ). The Delta rule can be formulated as…”

Section: Methodsmentioning

confidence: 99%

Utilizing the Switching Stochasticity of HfO2/TiOx-Based ReRAM Devices and the Concept of Multiple Device Synapses for the Classification of Overlapping and Noisy Patterns

et al. 2021

Self Cite

View full text Add to dashboard Cite

With the arrival of the Internet of Things (IoT) and the challenges arising from Big Data, neuromorphic chip concepts are seen as key solutions for coping with the massive amount of unstructured data streams by moving the computation closer to the sensors, the so-called “edge computing.” Augmenting these chips with emerging memory technologies enables these edge devices with non-volatile and adaptive properties which are desirable for low power and online learning operations. However, an energy- and area-efficient realization of these systems requires disruptive hardware changes. Memristor-based solutions for these concepts are in the focus of research and industry due to their low-power and high-density online learning potential. Specifically, the filamentary-type valence change mechanism (VCM memories) have shown to be a promising candidate In consequence, physical models capturing a broad spectrum of experimentally observed features such as the pronounced cycle-to-cycle (c2c) and device-to-device (d2d) variability are required for accurate evaluation of the proposed concepts. In this study, we present an in-depth experimental analysis of d2d and c2c variability of filamentary-type bipolar switching HfO2/TiOx nano-sized crossbar devices and match the experimentally observed variabilities to our physically motivated JART VCM compact model. Based on this approach, we evaluate the concept of parallel operation of devices as a synapse both experimentally and theoretically. These parallel synapses form a synaptic array which is at the core of neuromorphic chips. We exploit the c2c variability of these devices for stochastic online learning which has shown to increase the effective bit precision of the devices. Finally, we demonstrate that stochastic switching features for a pattern classification task that can be employed in an online learning neural network.

show abstract

Section: Methodsmentioning

confidence: 99%

Utilizing the Switching Stochasticity of HfO2/TiOx-Based ReRAM Devices and the Concept of Multiple Device Synapses for the Classification of Overlapping and Noisy Patterns

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The integration of CMOS technology with that of the emerging devices has been demonstrated for non-volatile filamentary switches [147] already at a commercial level [148]. There have also been some efforts in combining CMOS and memristor technologies to design supervised local error-based learning circuits using only one network layer by exploiting the properties of memristive devices [143], [149], [150].…”

Section: B Towards Edge Processing For Biomedical Applications With Neuromorphic Processorsmentioning

confidence: 99%

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

Azghadi

Lammie

Eshraghian

et al. 2020

IEEE Trans. Biomed. Circuits Syst.

Self Cite

138

View full text Add to dashboard Cite

The advent of dedicated Deep Learning (DL) accelerators and neuromorphic processors has brought on new opportunities for applying both Deep and Spiking Neural Network (SNN) algorithms to healthcare and biomedical applications at the edge. This can facilitate the advancement of medical Internet of Things (IoT) systems and Point of Care (PoC) devices. In this paper, we provide a tutorial describing how various technologies including emerging memristive devices, Field Programmable Gate Arrays (FPGAs), and Complementary Metal Oxide Semiconductor (CMOS) can be used to develop efficient DL accelerators to solve a wide variety of diagnostic, pattern recognition, and signal processing problems in healthcare. Furthermore, we explore how spiking neuromorphic processors can complement their DL counterparts for processing biomedical signals. The tutorial is augmented with case studies of the vast literature on neural network and neuromorphic hardware as applied to the healthcare domain. We benchmark various hardware platforms by performing a sensor fusion signal processing task combining electromyography (EMG) signals with computer vision. Comparisons are made between dedicated neuromorphic processors and embedded AI accelerators in terms of inference latency and energy. Finally, we provide our analysis of the field and share a perspective on the advantages, disadvantages, challenges, and opportunities that various accelerators and neuromorphic processors introduce to healthcare and biomedical domains.

show abstract

“…This running firing rate is registered at the end of the sequence and utilized in a logistic regression algorithm to calculate the output weights in the readout. The logistic regression could also be replaced with the online delta rule in an always-on fashion 40 . Such circuit implementation of the delta rule algorithm allows to train the output layer using Stochastic Gradient Descent (SGD) in a one-layer SNN.…”

Section: Static Networkmentioning

confidence: 99%

MEMSORN: Self-organization of an inhomogeneous memristive hardware for sequence learning

Payvand

Moro

Nomura

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Learning is a fundamental component for creating intelligent machines. Biological intelligence orchestrates synaptic and neuronal learning at multiple time-scales to self-organize populations of neurons for solving complex tasks. Inspired by this, we design and experimentally demonstrate an adaptive hardware architecture Memristive Self-organizing Spiking Recurrent Neural Network (MEMSORN). MEMSORN incorporates resistive memory (RRAM) in its synapses and neurons which configure their state based on Hebbian and Homeostatic plasticity respectively. For the first time, we derive these plasticity rules directly from the statistical measurements of our fabricated RRAM-based neurons and synapses. These “technologically plausible” learning rules exploit the intrinsic variability of the devices and improve the accuracy of the network on a sequence learning task by 30%. Finally, we compare the performance of MEMSORN to a fully-randomly set-up recurrent network on the same task, showing that self-organization improves the accuracy by more than 15%. This work demonstrates the importance of the device-circuit-algorithm co-design approach for implementing brain-inspired computing hardware.

show abstract

Analog Weight Updates with Compliance Current Modulation of Binary ReRAMs for On-Chip Learning

Cited by 21 publications

References 23 publications

Utilizing the Switching Stochasticity of HfO2/TiOx-Based ReRAM Devices and the Concept of Multiple Device Synapses for the Classification of Overlapping and Noisy Patterns

Utilizing the Switching Stochasticity of HfO2/TiOx-Based ReRAM Devices and the Concept of Multiple Device Synapses for the Classification of Overlapping and Noisy Patterns

Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications

MEMSORN: Self-organization of an inhomogeneous memristive hardware for sequence learning

Contact Info

Product

Resources

About