MITTS: Memory Inter-arrival Time Traffic Shaping

Zhou, Yanqi; Wentzlaff, David

doi:10.1109/isca.2016.53

Cited by 31 publications

(19 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Memory bandwidth regulation has been studied extensively at both hardware and software levels. Hardware-based techniques [23,24,25,33,75] can provide fine-grained memory bandwidth allocation to the cores using the memory controller; however, they require hardware customization and cannot be applied to COTS platforms. In contrast, software-based techniques [6,71,72,73] extend the system software to implement the memory bandwidth allocation; typically, they use the performance monitoring unit to monitor the memory requests from each core, and throttle a core when it exceeds its allocated memory bandwidth.…”

Section: Related Workmentioning

confidence: 99%

Holistic Resource Allocation for Multicore Real-Time Systems

Phan

Choi

et al. 2019

2019 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS)

View full text Add to dashboard Cite

This paper presents CaM, a holistic cache and memory bandwidth resource allocation strategy for multicore real-time systems. CaM is designed for partitioned scheduling, where tasks are mapped onto cores, and the shared cache and memory bandwidth resources are partitioned among cores to reduce resource interferences due to concurrent accesses. Based on our extension of LITMUS RT with Intel's Cache Allocation Technology and MemGuard, we present an experimental evaluation of the relationship between the allocation of cache and memory bandwidth resources and a task's WCET. Our resource allocation strategy exploits this relationship to map tasks onto cores, and to compute the resource allocation for each core. By grouping tasks with similar characteristics (in terms of resource demands) to the same core, it enables tasks on each core to fully utilize the assigned resources. In addition, based on the tasks' execution time behaviors with respect to their assigned resources, we can determine a desirable allocation that maximizes schedulability under resource constraints. Extensive evaluations using real-world benchmarks show that CaM offers near optimal schedulability performance while being highly efficient, and that it substantially outperforms existing solutions.

show abstract

Section: Related Workmentioning

confidence: 99%

Holistic Resource Allocation for Multicore Real-Time Systems

Phan

Choi

et al. 2019

2019 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS)

View full text Add to dashboard Cite

show abstract

“…In addition, with their mechanisms, the thread/CTA number, i.e., TLP, is not managed to achieve the optimal performance for GPUs. MITT [59] provides a fine-grain memory traffic shaping mechanism for cloud systems.…”

Section: Related Work Gpu Concurrent Kernel Executionmentioning

confidence: 99%

Coordinated CTA Combination and Bandwidth Partitioning for GPU Concurrent Kernel Execution

Lin

Dai

Mantor

et al. 2019

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Contemporary GPUs support multiple kernels to run concurrently on the same streaming multiprocessors (SMs). Recent studies have demonstrated that such concurrent kernel execution (CKE) improves both resource utilization and computational throughput. Most of the prior works focus on partitioning the GPU resources at the cooperative thread array (CTA) level or the warp scheduler level to improve CKE. However, significant performance slowdown and unfairness are observed when latency-sensitive kernels co-run with bandwidth-intensive ones. The reason is that bandwidth over-subscription from bandwidth-intensive kernels leads to much aggravated memory access latency, which is highly detrimental to latency-sensitive kernels. Even among bandwidth-intensive kernels, more intensive kernels may unfairly consume much higher bandwidth than less-intensive ones. In this article, we first make a case that such problems cannot be sufficiently solved by managing CTA combinations alone and reveal the fundamental reasons. Then, we propose a coordinated approach for CTA combination and bandwidth partitioning. Our approach dynamically detects co-running kernels as latency sensitive or bandwidth intensive. As both the DRAM bandwidth and L2-to-L1 Network-on-Chip (NoC) bandwidth can be the critical resource, our approach partitions both bandwidth resources coordinately along with selecting proper CTA combinations. The key objective is to allocate more CTA resources for latencysensitive kernels and more NoC/DRAM bandwidth resources to NoC-/DRAM-intensive kernels. We achieve it using a variation of dominant resource fairness (DRF). Compared with two state-of-the-art CKE optimization schemes, SMK [52] and WS [55], our approach improves the average harmonic speedup by 78% and 39%, respectively. Even compared to the best possible CTA combinations, which are obtained from an exhaustive search among all possible CTA combinations, our approach improves the harmonic speedup by up to 51% and 11% on average.

show abstract

“…In addition, new emerging data-intensive applications further increase memory traffic [4,5,47]. On the other hand, memory bandwidth is pin limited [4,65] and power constrained [45] and is therefore more difficult to scale [50]. More expensive, 3D-stacked DRAM technologies alleviate the bandwidth problem, but have capacity and power limitations [45].…”

Section: Introductionmentioning

confidence: 99%

MemSZ

Eldstål-Ahrens

Sourdis

2020

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

This article describes Memory Squeeze (MemSZ), a new approach for lossy general-purpose memory compression. MemSZ introduces a low latency, parallel design of the Squeeze (SZ) algorithm offering aggressive compression ratios, up to 16:1 in our implementation. Our compressor is placed between the memory controller and the cache hierarchy of a processor to reduce the memory traffic of applications that tolerate approximations in parts of their data. Thereby, the available off-chip bandwidth is utilized more efficiently improving system performance and energy efficiency. Two alternative multi-core variants of the MemSZ system are described. The first variant has a shared last-level cache (LLC) on the processor-die, which is modified to store both compressed and uncompressed data. The second has a 3D-stacked DRAM cache with larger cache lines that match the granularity of the compressed memory blocks and stores only uncompressed data. For applications that tolerate aggressive approximation in large fractions of their data, MemSZ reduces baseline memory traffic by up to 81%, execution time by up to 62%, and energy costs by up to 25% introducing up to 1.8% error to the application output. Compared to the current state-of-the-art lossy memory compression design, MemSZ improves the execution time, energy, and memory traffic by up to 15%, 9%, and 64%, respectively.

show abstract

MITTS: Memory Inter-arrival Time Traffic Shaping

Cited by 31 publications

References 40 publications

Holistic Resource Allocation for Multicore Real-Time Systems

Holistic Resource Allocation for Multicore Real-Time Systems

Coordinated CTA Combination and Bandwidth Partitioning for GPU Concurrent Kernel Execution

MemSZ

Contact Info

Product

Resources

About