Bandwidth-Aware Loop Tiling for DMA-Supported Scratchpad Memory

Abstract

Scratchpad Memory (SPM) is widely used in emerging domain-specific architectures and accelerators for improving energy efficiency and time predictability. Typically, SPM-based architectures use DMA for fetching data from off-chip memory and global load instructions for loading fine-grained data directly into registers. For such architectures, neither capacity-only nor bandwidthonly loop tiling can efficiently use the bandwidth and SPM. This paper introduces a bandwidth-aware loop tiling approach that enables a tradeoff between SPM space utilization and bandwidth utilization to be made, by leveraging a runtime tiling framework and a cross-host-kernel IPA. Experimental results demonstrate that our approach can achieve the performance improvement of up to 4x, with a geometric average of 26%.

Tile size selection of affine programs for GPGPUs using polyhedral cross-compilation

“…FlexTensor [66] is a framework that makes use of tiling and vectorization techniques to automatically explore and optimize tensor schedules on heterogeneous (CPU, GPU, and FPGA) systems. Wu et al [62] proposed a runtime tiling framework for scratchpad memories to achieve a balance between bandwidth and space utilization. Rawat et al [50] presented an automatic model-driven approach for code generation on GPUs guided by available resource information to adaptively implement fusion and time tiling.…”