RED: A ReRAM-based Deconvolution Accelerator

Fan, Zichen; Li, Ziru; Li, Bing; Chen, Yiran; Hai, Hai; Li, Qing

doi:10.23919/date.2019.8715103

Cited by 17 publications

(4 citation statements)

References 15 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Beyond convolutional computing engine, a number of works utilize ReRAM crossbars to support different computations and applications [24,[55][56][57]. For instance, Bojnordi et al [24] implement the restricted Boltzmann machine with ReRAM arrays.…”

Section: Matrix-vector Multiplications Isaacmentioning

confidence: 99%

An Overview of In-memory Processing with Emerging Non-volatile Memory for Data-intensive Applications

Yan

Hai

2019

Proceedings of the 2019 Great Lakes Symposium on VLSI

Self Cite

View full text Add to dashboard Cite

The conventional von Neumann architecture has been revealed as a major performance and energy bottleneck for rising data-intensive applications. The decade-old idea of leveraging in-memory processing to eliminate substantial data movements has returned and led extensive research activities. The effectiveness of in-memory processing heavily relies on the memory scalability, which cannot be satisfied by traditional memory technologies. Emerging non-volatile memories (eNVMs) that pose appealing qualities such as excellent scaling and low energy consumption, on the other hand, have been heavily investigated and explored for realizing in-memory processing architecture. In this paper, we summarize the recent research progress in eNVM-based in-memory processing from various aspects, including the adopted memory technologies, locations of the in-memory processing in system, supported arithmetics, as well as applied applications.

show abstract

Section: Matrix-vector Multiplications Isaacmentioning

confidence: 99%

An Overview of In-memory Processing with Emerging Non-volatile Memory for Data-intensive Applications

Yan

Hai

2019

Proceedings of the 2019 Great Lakes Symposium on VLSI

Self Cite

View full text Add to dashboard Cite

show abstract

“…al [26] proposed to accelerate TCONV on FPGA. Fcnengine [23] and Red [5] further realizes a fully convolutional accelerator that can handle both CONV and TCONV operations using unified processing elements (PEs). These work only focus on inference task but do not support training function.…”

Section: Related Work and Motivationmentioning

confidence: 99%

Zara

Chen

Song

et al. 2019

Proceedings of the 56th Annual Design Automation Conference 2019

Self Cite

View full text Add to dashboard Cite

Generative Adversarial Networks (GANs) recently demonstrated a great opportunity toward unsupervised learning with the intention to mitigate the massive human efforts on data labeling in supervised learning algorithms. GAN combines a generative model and a discriminative model to oppose each other in an adversarial situation to refine their abilities. Existing nonvolatile memory based machine learning accelerators, however, could not support the computational needs required by GAN training. Specifically, the generator utilizes a new operator, called transposed convolution, which introduces significant resource underutilization when executed on conventional neural network accelerators as it inserts massive zeros in its input before a convolution operation. In this work, we propose a novel computational deformation technique that synergistically optimizes the forward and backward functions in transposed convolution to eliminate the large resource underutilization. In addition, we present dedicated control units-a dataflow mapper and an operation scheduler, to support the proposed execution model with high parallelism and low energy consumption. ZARA is implemented with commodity ReRAM chips, and experimental results show that our design can improve GAN's training performance by averagely 1.6 × ∼23× over CMOS-based GAN accelerators. Compared to stateof-the-art ReRAM-based accelerator designs, ZARA also provides 1.15 × ∼2.1× performance improvement.

show abstract

“…[1]- [3], the data shuffling and the energy consumption while training GANs can be significantly reduced by performing in-memory VMM operations exploiting cross-point arrays of emerging non-volatile memories. Recently, several innovative deep convolutional (DC) GAN architectures including layer-wise pipelined computations [4], efficient deconvolutional operation [5], computational deformation technique to facilitate efficient utilization of computational resources in transpose convolution [6], and a ternary GAN [7] utilizing in-memory VMM engines based on RRAMs (with binary and 2-bit storage capability) and SOT-MRAMs were proposed. Moreover, a hybrid CMOSanalog RRAM-based implementation of DCGAN (without the pooling layer) including digital error propagation and weight update units was also proposed [8].…”

Section: Introductionmentioning

confidence: 99%

Energy-Efficient Implementation of Generative Adversarial Networks on Passive RRAM Crossbar Arrays

Sahay¹,

Nikam²,

Satyam³

2022

Preprint

View full text Add to dashboard Cite

Generative algorithms such as GANs are at the cusp of next revolution in the field of unsupervised learning and large-scale artificial data generation. However, the adversarial (competitive) co-training of the discriminative and generative networks in GAN makes them computationally intensive and hinders their deployment on the resource-constrained IoT edge devices. Moreover, the frequent data transfer between the discriminative and generative networks during training significantly degrades the efficacy of the von-Neumann GAN accelerators such as those based on GPU and FPGA. Therefore, there is an urgent need for development of ultra-compact and energy-efficient hardware accelerators for GANs. To this end, in this work, we propose to exploit the passive RRAM crossbar arrays for performing key operations of a fully-connected GAN: (a) true random noise generation for the generator network, (b) vector-by-matrix-multiplication with unprecedented energy-efficiency during the forward pass and backward propagation and (C) in-situ adversarial training using a hardware friendly Manhattan's rule. Our extensive analysis utilizing an experimentally calibrated phenomological model for passive RRAM crossbar array reveals an unforeseen trade-off between the accuracy and the energy dissipated while training the GAN network with different noise inputs to the generator. Furthermore, our results indicate that the spatial and temporal variations and true random noise, which are otherwise undesirable for memory application, boost the energy-efficiency of the GAN implementation on passive RRAM crossbar arrays without degrading its accuracy.

show abstract

RED: A ReRAM-based Deconvolution Accelerator

Cited by 17 publications

References 15 publications

An Overview of In-memory Processing with Emerging Non-volatile Memory for Data-intensive Applications

An Overview of In-memory Processing with Emerging Non-volatile Memory for Data-intensive Applications

Zara

Energy-Efficient Implementation of Generative Adversarial Networks on Passive RRAM Crossbar Arrays

Contact Info

Product

Resources

About