Heterogeneous system coherence for integrated CPU-GPU systems

Abstract

Many future heterogeneous systems will integrate CPUs and GPUs physically on a single chip and logically connect them via shared memory to avoid explicit data copying. Making this shared memory coherent facilitates programming and fine-grained sharing, but throughput-oriented GPUs can overwhelm CPUs with coherence requests not well-filtered by caches. Meanwhile, region coherence has been proposed for CPU-only systems to reduce snoop bandwidth by obtaining coherence permissions for large regions.This paper develops Heterogeneous System Coherence (HSC) for CPU-GPU systems to mitigate the coherence bandwidth effects of GPU memory requests. HSC replaces a standard directory with a region directory and adds a region buffer to the L2 cache. These structures allow the system to move bandwidth from the coherence network to the high-bandwidth direct-access bus without sacrificing coherence.Evaluation results with a subset of Rodinia benchmarks and the AMD APP SDK show that HSC can improve performance compared to a conventional directory protocol by an average of more than 2x and a maximum of more than 4.5x. Additionally, HSC reduces the bandwidth to the directory by an average of 94% and by more than 99% for four of the analyzed benchmarks.

Building Heterogeneous Unified Virtual Memories (UVMs) without the Overhead

“…To address the high bandwidth and MSRHs demands of future GPUs, HSC [Power et al 2013] uses a regional directory and buffers in both CPU and GPU L2 caches to filter directory probes. If the permission can be acquired by the region buffer, the requestor directly accesses the memory through a dedicated interconnect, otherwise it has to access the directory before accessing the memory.…”

“…The first step to make those systems more programming friendly is to provide a cache-coherent unified address space. Research proposals, either employ sequential consistency (SC) protocols to maintain coherence between devices (CPU-GPU) such as [Power et al 2013;Power et al 2015], or exchange some programming ease for simplicity and adopt release consistency (RC) such as [Singh et al 2013;. To maintain SC across devices, every write performed by a GPU core is eagerly made visible to the device and to the entire system.…”

“…Therefore, the CPU should avoid interaction with the GPU shared cache for latency purposes. The use of regional directories as proposed in heterogeneous system coher-ence (HSC) [Power et al 2013], significantly reduces the bandwidth and MSHRs pressure of these systems, but still maintains most of the complexity required to support SC, such as invalidations and indirections.…”

Minimizing the cost of iterative compilation with active learning

“…Even high-quality production compilers are out-of-date [31] and cannot extract the full performance potential of the hardware. The industry-wide trend towards heterogeneity only serves to make the optimization decision space even more complex, making effective heuristics near impossible to construct [44].…”

Fast Automatic Heuristic Construction Using Active Learning

“…Creating analytical models on which optimization heuristics can be based has become harder as processor complexity has increased, and this trend is bound to continue as processor designs move further towards heterogeneous parallelism [1]. Compiler developers often have to spend months if not years to get a heuristic right for a targeted architecture, and these days compilers often support a wide range of disparate processors.…”