Imbalanced cache partitioning for balanced data-parallel programs

Abstract

This paper investigates partitioning the ways of a shared last-level cache among the threads of a symmetric dataparallel application running on a chip-multiprocessor. Unlike prior work on way-partitioning for unrelated threads in a multiprogramming workload, the domain of multithreaded programs requires both throughput and fairness. Additionally, our workloads show no obvious thread differences to exploit: program threads see nearly identical IPC and data reuse as they progress (as expected for a well-written loadbalanced data-parallel program).Despite the balance and symmetry among threads, this paper shows that a balanced partitioning of cache ways between threads is suboptimal. Instead, this paper proposes a strategy of temporarily imbalancing the partitions between different threads to improve cache utilization by adapting to the locality behavior of the threads as captured by dynamic set-specific reuse-distance (SSRD). Cumulative SSRD histograms have knees that correspond to different important working sets; thus, cache ways can be taken away from a thread with only minimal performance impact if that thread is currently operating far from a knee. Those ways can then be given to a single "preferred" thread to push it over the next knee. The preferred thread is chosen in a round-robin fashion to ensure balanced progress over the execution. The algorithm also effectively handles scenarios where an unpartitioned cache might outperform any sort of explicit partitioning. This dynamic partition imbalance algorithm allows up to 44% reduction in execution time and 91% reduction in misses over an unpartitioned shared cache for 9 benchmarks from the PARSEC-2.0 and SPEC OMP suites.

Modeling cache performance beyond LRU

“…For instance, partitioning has been used to improve fairness [37,40], implement priorities and guarantee QoS [12,20,26], improve NUCA designs [3,33], and eliminate side-channel attacks [39]. Partitioning is therefore a general tool to help achieve system-level objectives.…”

“…Recall from Sec. 2.4 that partitioning has many benefits: improving shared cache performance [41], quality of service [26], fairness [40], security [39], etc.…”

“…Accurate predictions of cache behavior enable a large number of optimizations, including of single-threaded performance [5,15,38], shared cache performance [3,37,41,50], fairness [37,40], quality of service [3,26], security [39], etc. These optimizations can be performed in many ways, e.g.…”

Variation Aware Cache Partitioning for Multithreaded Programs

“…More recently, several efforts have investigated how cache partitioning can be applied to improve the overall throughput of multithreaded programs [17][18][19]. However, these techniques cannot directly address the impact of variations as they fail to take into account the performance imbalance in thread performance caused due to within-die variations.…”

Runtime-driven shared last-level cache management for task-parallel programs

“…Proposals for replacement policy modification have concentrated on tuning the replacement policy for better prediction of future reuse of cached data, with separate parameter tuning for each application [13,19,20,21,31]. Researchers have also proposed dynamic partitioning policies for multithreaded programs with static thread assignments and mapping, with an aim to ensure balanced progress for all threads while optimizing throughput [26,27].…”