RASA: Efficient Register-Aware Systolic Array Matrix Engine for CPU

Jeong, Geonhwa; Qin, Eric; Samajdar, Ananda; Hughes, Christopher J.; Subramoney, Sreenivas; Kim, Hyesoon; Krishna, Tushar

doi:10.1109/dac18074.2021.9586257

Cited by 9 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use a 32×16 conventional WS systolic array as the baseline, inspired by RASA [29] and Intel's TMUL [26]. Comparing against a dense matrix engine rather than a vector engine provides a strong baseline due to the huge gap in compute throughput between typical matrix engines and vector engines (Section III-A).…”

Section: E Vegeta Design Overviewmentioning

confidence: 99%

“…For modest-sized tiles, filling and draining the systolic array for a tile GEMM/SPMM instruction can significantly lower PE utilization. A recent work RASA [29] introduced pipelining the execution of tile GEMM instructions on a dense systolic array-based matrix engine; this overlaps different stages of execution for different instructions. We extend the pipelining concept and show how different tile SPMM instructions can be executed concurrently.…”

Section: Optimizationsmentioning

confidence: 99%

“…We set the frequency of the core as 2 GHz and fetch/issue/retire width as four with 16 III. We use reddish colors, black, greenish colors, and blue for dense matrix engines [26], [29], a design using STC-like config, config [39], VEGETA-S designs, and VEGETA-S with OF, respectively. pipeline stages, 97 ROB entries, and 96 load buffer entries.…”

Section: B Experimental Setupmentioning

confidence: 99%

“…Our work is orthogonal and complementary to those works since we are targeting a matrix engine that operates on tiles of the matrix instead of individual vectors. RASA [29] proposes control and data optimizations for CPU matrix engines to improve utilization via efficient pipelining and overlap. They divide a matrix multiplication with different sub-stages on the systolic array and introduce optimizations with pipelining and overlapping different stages.…”

Section: Related Workmentioning

confidence: 99%

“…Normalized runtime with different matrix engines in TableIII. We use reddish colors, black, greenish colors, and blue for dense matrix engines[26],[29], a design using STC-like config, config[39], VEGETA-S designs, and VEGETA-S with OF, respectively.…”

mentioning

confidence: 99%

See 4 more Smart Citations

VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUs

Jeong¹,

Damani²,

Rajeshkumar³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep Learning (DL) acceleration support in CPUs has recently gained a lot of traction, with several companies (Arm, Intel, IBM) announcing products with specialized matrix engines accessible via GEMM instructions. CPUs are pervasive and need to handle diverse requirements across DL workloads running in edge/HPC/cloud platforms. Therefore, as DL workloads embrace sparsity to reduce the computations and memory size of models, it is also imperative for CPUs to add support for sparsity to avoid under-utilization of the dense matrix engine and inefficient usage of the caches and registers. This work presents VEGETA, a set of ISA and microarchitecture extensions over dense matrix engines to support flexible structured sparsity for CPUs, enabling programmable support for diverse DL models with varying degrees of sparsity. Compared to the state-of-theart (SOTA) dense matrix engine in CPUs, a VEGETA engine provides 1.09×, 2.20×, 3.74×, and 3.28× speed-ups when running 4:4 (dense), 2:4, 1:4, and unstructured (95%) sparse DNN layers.

show abstract

Section: E Vegeta Design Overviewmentioning

confidence: 99%

Section: Optimizationsmentioning

confidence: 99%