Improving Resource Utilization in Heterogeneous CPU-GPU Systems

Abstract

Graphics processing units (GPUs) have attracted enormous interest over the past decade due to substantial increases in both performance and programmability. Programmers can potentially leverage GPUs for substantial performance gains, but at the cost of significant software engineering effort. In practice, most GPU applications do not effectively utilize all of the available resources in a system: they either fail to use use a resource at all or use a resource to less than its full potential. This underutilization can hurt both performance and energy efficiency. In this dissertation, we address the underutilization of resources in heterogeneous CPU-GPU systems in three different contexts.First, we address the underutilization of a single GPU by reducing CPU-GPU interaction to improve performance. We use as a case study a computationally-intensive video-tracking application from systems biology. Because of the high cost of CPU-GPU coordination, our initial, straightforward attempts to accelerate this application failed to effectively utilize the GPU. By leveraging some non-obvious optimization strategies, we significantly decreased the amount of CPU-GPU interaction and improved the performance of the GPU implementation by 26x relative to the best CPU implementation. Based on the lessons we learned, we present general guidelines for optimizing GPU applications as well as recommendations for system-level changes that would simplify the development of high-performance GPU applications.Next, we address underutilization at the system level by using load balancing to improve performance. We propose a dynamic scheduling algorithm that automatically and efficiently i Abstract ii divides the execution of a data-parallel kernel across multiple, possibly heterogeneous GPUs.We show that our scheduler can nearly match the performance of an unrealistic static scheduler when device performance is fixed, and can provide better performance when device performance varies.Finally, we address underutilization within a GPU by using frequency scaling to improve energy efficiency. We propose a novel algorithm for predicting the energy-optimal GPU clock frequencies for an arbitrary kernel. Using power measurements from real systems, we demonstrate that our algorithm improves significantly on the state of the art across multiple generations of GPUs. We also propose and evaluate techniques for decreasing the CPU's energy consumption during GPU computation.Many of the techniques presented in this dissertation can be used to improve the performance and energy efficiency of GPU applications with no programmer effort or software modifications required. As the diversity of available hardware systems continues to increase, automatic techniques such as these will become critical for software to fully realize the benefits of future hardware improvements.Approval Sheet

Performance and energy optimization of heterogeneous CPU-GPU systems for embedded applications

“…A dynamic scheduling algorithm was also proposed to balance an arbitrary GPU kernel across multiple devices and to keep each device fully utilized. A DVFS algorithm was also proposed to slow down the under-utilization of unutilized resources to increase effective utilization and energy efficiency [30].…”

Task Scheduling Frameworks for Heterogeneous Computing Toward Exascale

“…Consequently, they reduce the CPU/GPU interaction and synchronization aiming to improve the performance improves the energy consumption. Boyer [52] studied the iterative algorithms that are implemented and found that it causes the underutilization of the GPU resources, since they produce a high overhead of CPU/GPU arrangements. Thus, he presented several strategies to be applied when implementing such algorithms.…”