Thread Cluster Memory Scheduling: Exploiting Differences in Memory Access Behavior

Abstract

In a modern chip-multiprocessor system, memory is a shared resource among multiple concurrently executing threads. The memory scheduling algorithm should resolve memory contention by arbitrating memory access in such a way that competing threads progress at a relatively fast and even pace, resulting in high system throughput and fairness. Previously proposed memory scheduling algorithms are predominantly optimized for only one of these objectives: no scheduling algorithm provides the best system throughput and best fairness at the same time.This paper presents a new memory scheduling algorithm that addresses system throughput and fairness separately with the goal of achieving the best of both. The main idea is to divide threads into two separate clusters and employ different memory request scheduling policies in each cluster. Our proposal, Thread Cluster Memory scheduling (TCM), dynamically groups threads with similar memory access behavior into either the latency-sensitive (memory-non-intensive) or the bandwidth-sensitive (memory-intensive) cluster. TCM introduces three major ideas for prioritization: 1) we prioritize the latency-sensitive cluster over the bandwidth-sensitive cluster to improve system throughput; 2) we introduce a "niceness" metric that captures a thread's propensity to interfere with other threads; 3) we use niceness to periodically shuffle the priority order of the threads in the bandwidth-sensitive cluster to provide fair access to each thread in a way that reduces inter-thread interference. On the one hand, prioritizing memory-non-intensive threads significantly improves system throughput without degrading fairness, because such "light" threads only use a small fraction of the total available memory bandwidth. On the other hand, shuffling the priority order of memory-intensive threads improves fairness because it ensures no thread is disproportionately slowed down or starved.We evaluate TCM on a wide variety of multiprogrammed workloads and compare its performance to four previously proposed scheduling algorithms, finding that TCM achieves both the best system throughput and fairness. Averaged over 96 workloads on a 24-core system with 4 memory channels, TCM improves system throughput and reduces maximum slowdown by 4.6%/38.6% compared to ATLAS (previous work providing the best system throughput) and 7.6%/4.6% compared to PAR-BS (previous work providing the best fairness).

Bounding memory interference delay in COTS-based multi-core systems

“…These designs do not aim to provide worst-case bounds and can underestimate memory interference. Future memory controllers might incorporate ideas like batching and thread prioritization (e.g., [28,19]). This will lead to a different analysis, which could be interesting future work that builds on ours.…”

Improving DRAM performance by parallelizing refreshes with accesses

“…Each rank consists of multiple banks that share an internal bus for reading/writing data. 3 Because each bank acts as an independent entity, banks can serve multiple memory requests in parallel, offering banklevel parallelism [17,21,32]. A DRAM bank is further sub-divided into multiple subarrays [18,37,44] as shown in Figure 2.…”

Going vertical in memory management: Handling multiplicity by multi-policy

“…All programs are compiled by gcc 4.4.3 with the -O3 optimizations. Similar to previous work, we use weighted speedup [12] (WS) to measure system performance and maximum slowdown (MS) [12] for fairness: We compare several memory allocation schemes including the unmodified paging system in the Linux kernel, utilitybased partitioning [15], DRAM bank partitioning [16], random allocation [18] and our proposed HVR system. Figure 11 shows that for workloads that benefit from cacheonly and bank-only partitioning (50 workloads in Quadrant I of Figure 3), VP can accumulate the performance gains.…”

“…Previous research efforts [12,29] show that contention can significantly degrade the overall system performance and many solutions have been proposed to mitigate the contention problems.…”

“…More recently, cache partitioning is also adopted in heterogeneous GPU-CPU architectures to promote fair resource sharing among CPU and GPU applications [30], which exhibit drastically different memory access characteristics. Other efforts [3,9,12,15,25,28] classify workloads based on hardware profiling, and then choose appropriate scheduling policies for different classifications. OS-level approaches for memory utilization monitoring [6,7,26,27] have also been studied to provide knowledge for resource management.…”