Simultaneous Multikernel GPU: Multi-tasking throughput processors via fine-grained sharing

Wang, Zhenning; Yang, Jun; Melhem, Rami; Childers, Bruce R.; Zhang, Youtao; Guo, Minyi

doi:10.1109/hpca.2016.7446078

Cited by 121 publications

(85 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a recent study, Wang et al propose Simultaneous Multikernel (SMK), a fine‐grained dynamic sharing mechanism, that uses resources within a streaming multiprocessor by exploiting heterogeneity of different kernels. They evaluate their design using Parboil benchmarks and GPGPU‐Sim, a GPU simulator that they modify to support running multiple kernels on the same streaming multiprocessor.…”

Section: Surveymentioning

confidence: 99%

Methods for multitasking among real‐time embedded compute tasks running on the GPU

Muyan-Ozcelik

Owens

2017

Concurrency and Computation

View full text Add to dashboard Cite

Summary In this study, we provide an extensive survey on wide spectrum of scheduling methods for multitasking among graphics processing unit (GPU) computing tasks. We then design several schedulers and explain in detail the selected methods we have developed to implement our scheduling strategies. Next, we compare the performance of schedulers on various workloads running on Fermi and Kepler architectures and arrive at the following major conclusions: (1) Small kernels benefit from running kernels concurrently. (2) The combination of small kernels, high‐priority kernels with longer runtimes, and lower‐priority kernels with shorter runtimes benefits from a CPU scheduler that dynamically changes kernel order on the Fermi architecture. (3) Because of limitations of existing GPU architectures, currently CPU schedulers outperform their GPU counterparts. We also provide results and observations obtained from implementing and evaluating our schedulers on the NVIDIA Jetson TX1 system‐on‐chip architecture. We observe that although TX1 has the newer Maxwell architecture, the mechanism used for scheduler timings behaves differently on TX1 compared to Kepler leading to incorrect timings. In this paper, we describe our methods that allow us to report correct timings for CPU schedulers running on TX1. Finally, we propose new research directions involving the investigation of additional scheduling strategies.

show abstract

Section: Surveymentioning

confidence: 99%

Methods for multitasking among real‐time embedded compute tasks running on the GPU

Muyan-Ozcelik

Owens

2017

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Preemption mechanism [12] and dynamic scheduling [2] are orthogonal to our research, and therefore can be applied to our system. Recently, simultaneous multi-kernel, (SMK) which executes multiple kernels within a same SM, is proposed to improve the utilization of resources inside SMs [15,16]. Though these hardware algorithms can optimize thread block allocation for high resource utilization inside SMs, the control logic is too complex.…”

Section: Case Study: Pf+bp and Hs+smmentioning

confidence: 99%

Efficient GPU multitasking with latency minimization and cache boosting

Kim

Chu

Park

2017

IEICE Electron. Express

View full text Add to dashboard Cite

GPU spatial multitasking has been proven to be quite effective at executing different applications concurrently using SM partitioning. However, while it maximizes total throughput, latency-critical applications often cannot meet their deadlines due to the increased execution time. Furthermore, SM partitioning cannot allocate the appropriate L1 cache size per kernel. To solve these problems, this paper proposes a new application-aware resource allocation framework called GPU Fine-Tuner, for assigning appropriate resources to GPU kernels. To minimize the execution time of latencyconstrained applications, it assigns them more SMs when performance is not affected. It also increases the cache size of SMs for cache-sensitive kernels using resource borrowing from neighbors for cache-insensitive kernels. Experimental results show that the Fine-Tuner outperforms GPU spatial multitasking with up to 15% less average latency without performance degradation.

show abstract

“…Lin and others have proposed the use of compression to reduce kernel switch overhead [5]. Wang and others [23] have designed simultaneous multi-kernel (SMK), a hardware method to support incremental kernel evictions. Although these schemes have shown high promises, they require new hardware support.…”

Section: Related Workmentioning

confidence: 99%

EffiSha

et al. 2017

View full text Add to dashboard Cite

Modern GPUs are broadly adopted in many multitasking environments, including data centers and smartphones. However, the current support for the scheduling of multiple GPU kernels (from different applications) is limited, forming a major barrier for GPU to meet many practical needs. This work for the first time demonstrates that on existing GPUs, efficient preemptive scheduling of GPU kernels is possible even without special hardware support. Specifically, it presents EffiSha, a pure software framework that enables preemptive scheduling of GPU kernels with very low overhead. The enabled preemptive scheduler offers flexible support of kernels of different priorities, and demonstrates significant potential for reducing the average turnaround time and improving the system overall throughput of programs that time share a modern GPU.

show abstract

Simultaneous Multikernel GPU: Multi-tasking throughput processors via fine-grained sharing

Cited by 121 publications

References 17 publications

Methods for multitasking among real‐time embedded compute tasks running on the GPU

Methods for multitasking among real‐time embedded compute tasks running on the GPU

Efficient GPU multitasking with latency minimization and cache boosting

EffiSha

Contact Info

Product

Resources

About