Learning Sparse Matrix Row Permutations for Efficient SpMM on GPU Architectures

Mehrabi, Atefeh; Lee, Donghyuk; Chatterjee, Niladrish; Sorin, Daniel J.; Lee, Benjamin C.; O'Connor, Mike

doi:10.1109/ispass51385.2021.00016

Cited by 11 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, the sparsity of TWNs is not well utilized. The sparse weights of TWNs bring the need for compressed sparse formats to save storage [22], [23]. However, the compression and decompression modules will increase the complexity of the accelerator design, the area cost, and the power.…”

Section: Introductionmentioning

confidence: 99%

FAT: An In-Memory Accelerator With Fast Addition for Ternary Weight Neural Networks

Zhu

Duong

Chen

et al. 2023

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) demonstrate excellent performance in various applications but have high computational complexity. Quantization is applied to reduce the latency and storage cost of CNNs. Among the quantization methods, Binary and Ternary Weight Networks (BWNs and TWNs) have a unique advantage over 8-bit and 4-bit quantization. They replace the multiplication operations in CNNs with additions, which are favoured on In-Memory-Computing (IMC) devices. IMC acceleration for BWNs has been widely studied. However, though TWNs have higher accuracy and better sparsity than BWNs, IMC acceleration for TWNs has limited research. TWNs on existing IMC devices are inefficient because the sparsity is not well utilized, and the addition operation is not efficient.In this paper, we propose FAT as a novel IMC accelerator for TWNs. First, we propose a Sparse Addition Control Unit, which utilizes the sparsity of TWNs to skip the null operations on zero weights. Second, we propose a fast addition scheme based on the memory Sense Amplifier to avoid the time overhead of both carry propagation and writing back the carry to memory cells. Third, we further propose a Combined-Stationary data mapping to reduce the data movement of activations and weights and increase the parallelism across memory columns. Simulation results show that for addition operations at the Sense Amplifier level, FAT achieves 2.00× speedup, 1.22× power efficiency and 1.22× area efficiency compared with a State-Of-The-Art IMC accelerator ParaPIM. FAT achieves 10.02× speedup and 12.19× energy efficiency compared with ParaPIM on networks with 80% average sparsity.

show abstract

Section: Introductionmentioning

confidence: 99%

FAT: An In-Memory Accelerator With Fast Addition for Ternary Weight Neural Networks

Zhu

Duong

Chen

et al. 2023

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

show abstract

“…Various sparse matrix storage formats have been proposed to reduce the memory and computation overhead of SpMM [14,19]. Studies have also shown that choosing the right storage format can have a significant impact on the SpMM performance [21]. Although SpMM performance optimization is a well-studied field in traditional high-performance computing (HPC) domains, the benefit of sparse matrix storage format selection is unclear on the new GNN workloads.…”

Section: Introductionmentioning

confidence: 99%

Optimizing Sparse Matrix Multiplications for Graph Neural Networks

Qiu

Liang

Wang

2022

Languages and Compilers for Parallel Computing

View full text Add to dashboard Cite

show abstract

“…First, the sparsity of TWNs is not well utilized. The sparse weights of TWNs bring the need for compressed sparse formats to save storage [162,163]. However, the compression and decompression modules will increase the complexity of the accelerator design, the area cost, and the power.…”

Section: Introductionmentioning

confidence: 99%

Deep learning acceleration: from quantization to in-memory computing

Zhu¹

View full text Add to dashboard Cite

He guided me from an undergraduate student to becoming a graduate researcher.His emphasis on research innovation, technical contribution, and persuasive writing greatly benefited me. Furthermore, he let me explore the research areas I am interested in and pushed me to publish papers in time. I wish him a happy life and academic prosperity. At the same time, I would like to thank my thesis advisory committee, Prof. Xueyan Tang and Dr. Tao Luo, for their insightful comments and support of my research.

show abstract

Learning Sparse Matrix Row Permutations for Efficient SpMM on GPU Architectures

Cited by 11 publications

References 26 publications

FAT: An In-Memory Accelerator With Fast Addition for Ternary Weight Neural Networks

FAT: An In-Memory Accelerator With Fast Addition for Ternary Weight Neural Networks

Optimizing Sparse Matrix Multiplications for Graph Neural Networks

Deep learning acceleration: from quantization to in-memory computing

Contact Info

Product

Resources

About