Dymaxion++: A Directive-Based API to Optimize Data Layout and Memory Mapping for Heterogeneous Systems

Abstract

Abstract-There has been a growing trend in using heterogeneous systems with CPUs and GPUs to solve diverse compute problems. However, high application performance on these platforms relies on efficient memory accesses. For many applications, CPUs and GPUs prefer different memory mappings and datastructure layouts. This in turn requires developers to use devicespecific strategies for memory access optimizations. Achieving both code and performance portability becomes a challenge for heterogeneous computing.This paper proposes a directive-based API, Dymaxion++, which enables programmers to optimize memory access patterns across devices with a simple interface. Use of Dymaxion++ requires only minimal modifications to existing codes with a small set of pragma extensions. The current framework augments the original Dymaxion framework [6] with a clean abstraction backed by a source-to-source code translator. Dymaxion++ also provides additional programming features to map data structures to GPU's hybrid memory spaces (e.g. texture and constant memory) for different uses. Additionally, data layout transformation is enabled while exchanging data between GPU scratchpad and device memory as well as between system memory and device memory.

Automatic Data Layout Optimizations for GPUs

“…It extends NVIDIA's CUDA API with a data index transformation and a latency hiding mechanism based on CUDA stream. Dymaxion C++ [3] further extends prior work. However, it does not relieve the programmer from selecting a good data layout.…”