Memristor crossbar-based ultra-efficient next-generation baseband processors

Yuan, Geng; Ding, Caiwen; Cai, Ruizhe; Ma, Xiaolong; Zhao, Ziyi; Ren, Ao; Yuan, Bo; Wang, Yanzhi

doi:10.1109/mwscas.2017.8053125

Cited by 27 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They can be formulated as Eqn. (5). And M and N stands for the bit-length of inputs and weights respectively.…”

Section: Sot-mram Processing-in-memory Engine For Dnnmentioning

confidence: 99%

“…Large-scale DNNs achieve significant improvement in many challenging problems, such as image classification [1], speech recognition [2] and natural language processing [3]. However, as the number of layers and the layer size are both expanding, the introduced intensive computation and storage have brought challenges to the traditional Von-Neumann architecture [4], such as computing wall, massive data movement and high power consumption [5], [6]. Furthermore, the deep structure and large model size will make DNNs prohibitive to embedded systems and IoT devices, where low power consumption are required.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A SOT-MRAM-based Processing-In-Memory Engine for Highly Compressed DNN Implementation

Yuan,

Ma,

Lin

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

“…They can be formulated as Eqn. (5). And M and N stands for the bit-length of inputs and weights respectively.…”

Section: Sot-mram Processing-in-memory Engine For Dnnmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A SOT-MRAM-based Processing-In-Memory Engine for Highly Compressed DNN Implementation

Yuan,

Ma,

Lin

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

“…The improvement is statistically achieved for all models, which avoids conducting optimizations for each individual device. Resistive RAM (ReRAM), as an emerging memory device, has many promising features, including non-volatility, nearly zero leakage power, high integration density, and high scalability [10], [11]. Every memory cell comprises a ReRAM element to store one or more data bits.…”

Section: Introductionmentioning

confidence: 99%

Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI

Yuan¹,

Liao²,

Ma³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recent research demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector multiplication-the intensive and key computation in deep neural networks (DNNs). However, hardware failure, such as stuck-at-fault defects, is one of the main concerns that impedes the ReRAM devices to be a feasible solution for real implementations. The existing solutions to address this issue usually require an optimization to be conducted for each individual device, which is impractical for mass-produced products (e.g., IoT devices). In this paper, we rethink the value of weight pruning in ReRAM-based DNN design from the perspective of model fault tolerance. And a differential mapping scheme is proposed to improve the fault tolerance under a high stuck-on fault rate. Our method can tolerate almost an order of magnitude higher failure rate than the traditional two-column method in representative DNN tasks. More importantly, our method does not require extra hardware cost compared to the traditional two-column mapping scheme. The improvement is universal and does not require the optimization process for each individual device.

show abstract

“…vector multiplication in the analog domain and the computation is in O(1) time complexity [12,13]. Motivated by the fact that there is no precedent model that is structured pruned and quantized as well as satisfying memristor hardware constraints, in this work, a memristor-based ADMM regularized optimization method is utilized both on structured pruning and weight quantization in order to mitigate the accuracy degradation during extreme model compression.…”

Section: Introductionmentioning

confidence: 99%

Tiny but Accurate: A Pruned, Quantized and Optimized Memristor Crossbar Framework for Ultra Efficient DNN Implementation

Yuan

Lin

et al. 2020

2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC)

Self Cite

View full text Add to dashboard Cite

The state-of-art DNN structures involve intensive computation and high memory storage. To mitigate the challenges, the memristor crossbar array has emerged as an intrinsically suitable matrix computation and low-power acceleration framework for DNN applications. However, the high accuracy solution for extreme model compression on memristor crossbar array architecture is still waiting for unraveling. In this paper, we propose a memristor-based DNN framework which combines both structured weight pruning and quantization by incorporating alternating direction method of multipliers (ADMM) algorithm for better pruning and quantization performance. We also discover the non-optimality of the ADMM solution in weight pruning and the unused data path in a structured pruned model. Motivated by these discoveries, we design a software-hardware co-optimization framework which contains the first proposed Network Purification and Unused Path Removal algorithms targeting on post-processing a structured pruned model after ADMM steps. By taking memristor hardware constraints into our whole framework, we achieve extreme high compression ratio on the state-of-art neural network structures with minimum accuracy loss. For quantizing structured pruned model, our framework achieves nearly no accuracy loss after quantizing weights to 8-bit memristor weight representation. We share our models at anonymous link https://bit. ly/2VnMUy0.

show abstract

Memristor crossbar-based ultra-efficient next-generation baseband processors

Cited by 27 publications

References 19 publications

A SOT-MRAM-based Processing-In-Memory Engine for Highly Compressed DNN Implementation

A SOT-MRAM-based Processing-In-Memory Engine for Highly Compressed DNN Implementation

Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI

Tiny but Accurate: A Pruned, Quantized and Optimized Memristor Crossbar Framework for Ultra Efficient DNN Implementation

Contact Info

Product

Resources

About