Perspective on training fully connected networks with resistive
                    memories: Device requirements for multiple conductances of varying
                    significance

Cristiano, Giorgio; Giordano, Massimo; Ambrogio, Stefano; Romero, Louis P.; Cheng, C.-W.; Narayanan, Pritish; Tsai, Hsinyu; Shelby, R. M.; Burr, Geoffrey W.

doi:10.1063/1.5042462

Cited by 33 publications

(23 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The only exception is represented by novel Li-ion devices, which appear to be very promising, with a simulated performance of around 98% [119], even though the necessary technology maturity and high-density integration have not been reached yet. Alternatively, more complex structures, including multiple pair of memristive devices, such as PCM and RRAM, could mitigate the need for high linearity, but at the expense of a lower integration density [176].…”

Section: Discussionmentioning

confidence: 99%

Memristive and CMOS Devices for Neuromorphic Computing

et al. 2020

View full text Add to dashboard Cite

Neuromorphic computing has emerged as one of the most promising paradigms to overcome the limitations of von Neumann architecture of conventional digital processors. The aim of neuromorphic computing is to faithfully reproduce the computing processes in the human brain, thus paralleling its outstanding energy efficiency and compactness. Toward this goal, however, some major challenges have to be faced. Since the brain processes information by high-density neural networks with ultra-low power consumption, novel device concepts combining high scalability, low-power operation, and advanced computing functionality must be developed. This work provides an overview of the most promising device concepts in neuromorphic computing including complementary metal-oxide semiconductor (CMOS) and memristive technologies. First, the physics and operation of CMOS-based floating-gate memory devices in artificial neural networks will be addressed. Then, several memristive concepts will be reviewed and discussed for applications in deep neural network and spiking neural network architectures. Finally, the main technology challenges and perspectives of neuromorphic computing will be discussed.

show abstract

Section: Discussionmentioning

confidence: 99%

Memristive and CMOS Devices for Neuromorphic Computing

et al. 2020

View full text Add to dashboard Cite

show abstract

“…[28] The model captures the "s-shaped" conductance versus pulse profile that is typical in PCM as well as interdevice (device-to-device) and intradevice (cycle-to-cycle) variations. [28] The model captures the "s-shaped" conductance versus pulse profile that is typical in PCM as well as interdevice (device-to-device) and intradevice (cycle-to-cycle) variations.…”

Section: Pcm Conductance Modelmentioning

confidence: 99%

“…[28] The model captures the "s-shaped" conductance versus pulse profile that is typical in PCM as well as interdevice (device-to-device) and intradevice (cycle-to-cycle) variations. To describe the shape of the conductance increase, a "Jump Table" [28] approach is employed (see Figure 2a). To describe the shape of the conductance increase, a "Jump Table" [28] approach is employed (see Figure 2a).…”

Section: Pcm Conductance Modelmentioning

confidence: 99%

“…[8,28] The algorithm selects a row and stores the corresponding target weights in the peripheral circuitry at the north side. [8,28] The algorithm selects a row and stores the corresponding target weights in the peripheral circuitry at the north side.…”

Section: Wwwadvelectronicmatdementioning

confidence: 99%

“…A "Jump Table" construct [28] is used to accurately model the conductance evolution of the NVM devices, using normally distributed random variables to introduce both cycle-to-cycle and device-to-device variability in the size of conductance jumps, as well as device-to-device variability in maximum conductance. This paper quantifies P fail given one particular weight-and conductance-programming strategy, and describes the relationship between P fail and programming speed as controlled by choice of the gain factor F and other parameters.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Weight Programming in DNN Analog Hardware Accelerators in the Presence of NVM Variability

Mackin

Tsai

Ambrogio

et al. 2019

Adv Elect Materials

Self Cite

View full text Add to dashboard Cite

large and unavoidable time and energy penalties for data transport between memory and computational blocks-the so-called "von Neumann" bottleneck.Analog hardware accelerators for deep learning can avoid this bottleneck by performing MAC operations in memory using a crossbar array structure. [5][6][7] In these crossbar arrays, nonvolatile memory (NVM) elements are used to encode synaptic weights. This was recently shown capable of 280× speedup in per area throughput while also providing 100× enhancement in per area energy efficiency over state-of-the-art GPUs. [8] While the benefits of speed are readily appreciated, enhancement in energy efficiency can be a powerful driver of purchasing decisions in the data center space as well. [9][10][11][12] Crossbar arrays have been implemented using a variety of analog NVM elements, including resistive RAM (ReRAM), [13,14] conductive-bridging RAM (CBRAM), [15] flash, [16][17][18][19][20] and phase-change memory (PCM). [21,22] However, most NVM device candidates exhibit varying extents of nonideal behavior, including limited resistance contrast, significant nonlinearity in conductance change, as well as strong asymmetry in bidirectional programming. At the application level, this manifests into neural network accuracies that are significantly lower than software-based approaches. Many of these nonidealities, however, can be addressed to an extent through device and circuit-level engineering. One key idea is multiple conductances of varying significance (Figure 1a), where each synaptic weight is distributed across at least two conductance pairs using W = F(G + − G − ) + (g + − g − ). For instance, during training, such a PCM cell could be paired with a capacitive cell to combine complementary features of both and relax the overall device requirements on PCM. [8] In such a scheme, the majority of DNN weight tuning can occur on a linear but volatile 3T1C memory structure, with periodic but infrequent transfer to the PCM cell for nonvolatile storage. Row-(or column-)wise weight transfer also arises in ex situ training and other training variants. Since PCM is subject to the previously mentioned nonidealities, this weight transfer is best performed with a closed-loop iterative tuning procedure with multiple write-plus-read-verify steps to achieve the overall target weight. To avoid performance degradation, particularly during training, NVM programming Crossbar arrays of nonvolatile memory (NVM) can potentially accelerate development of deep neural networks (DNNs) by implementing crucial multiply-accumulate (MAC) operations at the location of data. Effective weight-programming procedures can both minimize the performance impact during training and reduce the down time for inference, where new parameter sets may need to be loaded. Simultaneous weight programming along an entire dimension (e.g., row or column) of a crossbar array in the context of forward inference and training is shown to be important. A framework for determining the optimal hardware conditions in which to program w...

show abstract

Recent Advancements in Emerging Neuromorphic Device Technologies

Woo

Kim

et al. 2020

Advanced Intelligent Systems

View full text Add to dashboard Cite

The explosive growth of data and information has motivated technological developments in computing systems that utilize them for efficiently discovering patterns and gaining relevant insights. Inspired by the structure and functions of biological synapses and neurons in the brain, neural network algorithms that can realize highly parallel computations have been implemented on conventional silicon transistor‐based hardware. However, synapses composed of multiple transistors allow only binary information to be stored, and processing such digital states through complicated silicon neuron circuits makes low‐power and low‐latency computing difficult. Therefore, the attractiveness of the emerging memories and switches for synaptic and neuronal elements, respectively, in implementing neuromorphic systems, which are suitable for performing energy‐efficient cognitive functions and recognition, is discussed herein. Based on a literature survey, recent progress concerning memories shows that novel strategies related to materials and device engineering to mitigate challenges are presented to primarily achieve nonvolatile analog synaptic characteristics. Attempts to emulate the role of the neuron in various ways using compact switches and volatile memories are also discussed. It is hoped that this review will help direct future interdisciplinary research on device, circuit, and architecture levels of neuromorphic systems.

show abstract

Perspective on training fully connected networks with resistive memories: Device requirements for multiple conductances of varying significance

Cited by 33 publications

References 19 publications

Memristive and CMOS Devices for Neuromorphic Computing

Memristive and CMOS Devices for Neuromorphic Computing

Weight Programming in DNN Analog Hardware Accelerators in the Presence of NVM Variability

Recent Advancements in Emerging Neuromorphic Device Technologies

Contact Info

Product

Resources

About