Page Placement Strategies for GPUs within Heterogeneous Memory Systems

Abstract

Systems from smartphones to supercomputers are increasingly heterogeneous, being composed of both CPUs and GPUs. To maximize cost and energy efficiency, these systems will increasingly use globally-addressable heterogeneous memory systems, making choices about memory page placement critical to performance. In this work we show that current page placement policies are not sufficient to maximize GPU performance in these heterogeneous memory systems. We propose two new page placement policies that improve GPU performance: one application agnostic and one using application profile information. Our application agnostic policy, bandwidth-aware (BW-AWARE) placement, maximizes GPU throughput by balancing page placement across the memories based on the aggregate memory bandwidth available in a system. Our simulation-based results show that BW-AWARE placement outperforms the existing Linux INTERLEAVE and LOCAL policies by 35% and 18% on average for GPU compute workloads. We build upon BW-AWARE placement by developing a compilerbased profiling mechanism that provides programmers with information about GPU application data structure access patterns. Combining this information with simple program-annotated hints about memory placement, our hint-based page placement approach performs within 90% of oracular page placement on average, largely mitigating the need for costly dynamic page tracking and migration.

Thread Batching for High-performance Energy-efficient GPU Memory Design

“…• BW-AWARE denotes a synergistic bandwidth-aware page placement policy in [2]. It places the GPU pages across the heterogeneous memory system, i.e., GDDR5 and DDR3 DRAM, and their memory bandwidth is shared across GPU pages.…”

“…Because of discrepant bandwidth requirements, it is anticipated that GPU will be still physically attached with bandwidth-optimized DRAM, while CPU is attached with capacity-and cost-optimized DRAM. DRAMs of GPU and CPU share a unified memory address space [2]. In such heterogeneous cache coherent non-uniform memory access (CC-NUMA) system, a computing unit has different access delays to local and remote memories even it sees a unified address space.…”

“…We compare TEMP and TBAS with some representative thread scheduling techniques, including the cache-conscious wavefront scheduler (CCWS) [37], OWL [18] and the bandwidth-aware policy (BW-AWARE) [2]. We set CCWS as our baseline and integrate OWL, BW-AWARE and our techniques on top of CCWS.…”

Design and implementation of bandwidth-aware memory placement and migration policies for heterogeneous memory systems

“…There are many previous works [2,12,13,14] on heterogeneous memory systems where DRAM and NVM coexist. Kannan et al particularly allowed to manage DRAM and NVM in a single virtual space and studied how NVM can be used to perform tasks that require persistence attributes.…”

“…However, they did not study how to maximize the bandwidth of different types of memory clusters on the system. The most similar previous work was that of [12]. This work studied the bandwidth-aware memory placement policy that considers the CPU and GPU bandwidths.…”