Demonstration of Generative Adversarial Network by Intrinsic Random Noises of Analog RRAM Devices

Lin, Yudeng; Wu, Huaqiang; Gao, Bin; Yao, Peng; Wu, Wei; Zhang, Qingtian; Zhang, Xiang; Li, Fuhai; Lu, Jiwu; Li, Gezi; Yu, Shimeng; Qian, He

doi:10.1109/iedm.2018.8614483

Cited by 28 publications

(22 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Utilizing the benefits of a passive RRAM crossbar array in performing in-situ computations, a vanilla GAN implementa- tion was performed, similar to [5], to synthesize handwritten digits from the MNIST dataset.…”

Section: Modeling and Simulationmentioning

confidence: 99%

Energy-Efficient Implementation of Generative Adversarial Networks on Passive RRAM Crossbar Arrays

Satyam¹,

Nikam²,

Sahay³

2021

Preprint

View full text Add to dashboard Cite

There has been immense development in the area of generative algorithms in recent years. Contrary to the discriminative models, which map high dimensional inputs to class labels, generative models are used in Variational autoencoders (VAEs) and Generative Adversarial Networks (GANs). Extensive studies have been done to improve unsupervised learning and GANs are one of the most successful algorithms to come up in the domain. With the benefits of providing greater accuracies, GANs have been expensive in terms of energy and speed, due to the vast number of Vector Matrix Multiplications computed on a large weight matrix. To overcome this hurdle, several works have been done on GPU and FPGA based accelerators. However, the Von Neumann bottleneck limits the accuracies and energy efficiency one can achieve and so Neuromorphic computing has been adopted greatly to exceed these limits. In this work, we have proposed an implementation of GANs on passive memristor crossbar arrays. We have performed a fixed amplitude training to update the weights with the crossbar as the backend. We also proposed to use a true random noise for the network. The simulation results show that our implementation has low energy consumption with comparable accuracies to the software counterpart.

show abstract

Section: Modeling and Simulationmentioning

confidence: 99%

Energy-Efficient Implementation of Generative Adversarial Networks on Passive RRAM Crossbar Arrays

Satyam¹,

Nikam²,

Sahay³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…当前用忆阻器阵列实现基础的人工神经网络算法已经卓有成效. 按神经网络的训练方法分, 在无监督学习上已实现主成分分析 [28] 、稀疏编码 [29] 、联想学习 [30] 、生成对抗网络 [31] 等; 在监督学习上已实现单层感知机、多层感知机 [32] 、卷积神经网络 [26] 、长短期记忆网络 [33] 、循环卷积神经网络 [34] 等; 研究者们还在阵列上利用不同方式实现了强化学习 [35,36] . 2019年, 密歇根大学研究团队制作出通用可编程忆阻器: CMOS混合架构加速器芯片 [37] , 阵列规模为54×108, 推动了忆阻 [39] ; 而大脑在感知和学习的过程中工作在混沌的边缘 [40] , 具有强大的全局搜索能力和实时计算能力 [41] .…”

Section: 效应可通过加入选择器改善例如加入晶体管作为选unclassified

Progresses and outlook in neuromorphic devices

et al. 2019

View full text Add to dashboard Cite

“…[10] On the contrary, some attempts have been devoted to utilize noise characteristics of the device instead as computing resources. For instance, the intrinsic random noise of RRAM devices can be utilized as the input of a generative adversarial network, [11] which is helpful for mitigating mode collapse problems when training the network. Similarly, the intrinsic analog noise in RRAM crossbars can be leveraged for the implementation of a simulated annealing algorithm.…”

mentioning

confidence: 99%

Artificial Intelligence Goes Physical

Jing

Yang

2021

Small Science

View full text Add to dashboard Cite

Exploiting the intrinsic nonlinearity in physical reservoirs, e.g., dopant-atom networks, provides a new approach toward highly efficient computing such as feature projection and classification. In a recent study by Chen et al., the computational capability of dopant-atom network was investigated and found to diminish as the signal-to-noise ratio (SNR) increased, indicating the existence of an optimal bias condition. Although high SNR is often pursued in signal processing, it shows that embracing noise in non-conventional computing systems may lead to a leap in computing capacity. This work showcased that material or device physics in different domains offer valuable substrates for complex computing functions and high energy efficiency.

show abstract

Demonstration of Generative Adversarial Network by Intrinsic Random Noises of Analog RRAM Devices

Cited by 28 publications

References 3 publications

Energy-Efficient Implementation of Generative Adversarial Networks on Passive RRAM Crossbar Arrays

Energy-Efficient Implementation of Generative Adversarial Networks on Passive RRAM Crossbar Arrays

Progresses and outlook in neuromorphic devices

Artificial Intelligence Goes Physical

Contact Info

Product

Resources

About