Automatic Optimization of Thread Mapping for a GPGPU Programming Framework

Abstract

Although General Purpose computation on Graphics Processing Units (GPGPU) is widely used for the high-performance computing, standard programming frameworks such as CUDA and OpenCL are still difficult to use. They require low-level specifications and the handoptimization is a large burden. Therefore we are developing an easier framework named MESI-CUDA. Based on a virtual shared memory model, MESI-CUDA hides low-level memory management and data transfer from the user. The compiler generates low-level code and also optimizes memory accesses applying conventional hand-optimizing techniques. However, creating GPU threads is same as CUDA; the user specifies thread mapping, i.e. thread indexing and the size of thread blocks run on each streaming multiprocessors (SM). The mapping largely affects the execution performance and may obstruct automatic optimization of MESI-CUDA compiler. Therefore, the user must find optimal specification considering physical parameters. In this paper, we propose a new thread mapping scheme. We introduce new thread creation syntax specifying hardware-independent logical mapping, which is converted into optimized physical mapping at compile time. Making static analysis of array index expressions, we obtain groups of threads accessing the same or neighboring array elements. Mapping such threads into the same thread block and assigning consecutive thread indices, the physical mapping is determined to maximize the effect of memory access optimization. As the result of evaluation, our scheme could find optimal mapping strategies for five benchmark programs. Memory access transactions were reduced to approximately 1/4 and 1.4-76 times speedup is achieved compared with the worst mapping.

Dynamic Task Scheduling Scheme for a GPGPU Programming Framework

“…Although such low-level API enables hand-tuning considering hardware specifications, it is difficult and not performance portable. Therefore we are developing an easier programming framework MESI-CUDA [5,6,7,8].…”

“…We regard the difference of characteristics between CPU and CUDA cores is not negligible on HPC programming. However, we hide SMs by introducing a logical specification of thread mapping [8] and mapping optimization by the compiler.…”

“…To achieve high-performance without user's hand-tuning, the compiler performs optimizations based on the static analysis. For this purpose, we have developed automatic optimization schemes such as overlapping thread executions and data transfers [5], explicit cache using shared memories [7], and thread mappings improving the device memory accesses [8].…”

“…Adopting a single and consecutive segment, the assignment to each task can be expressed using two integer values b, s, which are the starting block index and the number of blocks of the segment, respectively. In addition, assigning neighboring blocks to a task can expect smaller access ranges in the task because our current scheme optimizes the thread mapping to improve access locality on VS variables [8]. On the other hand, the size of each task, i. e. the number of assigned thread blocks, can be tuned for the optimization.…”

“…Moreover, the current version does not use multiple GPUs automatically [4]. To handle large-scale problems without any additional user code, we propose a dynamic scheduling scheme introducing implicit tasks for our CUDA variation MESI-CUDA [5,6,7,8]. MESI-CUDA is designed to abstract the low-level GPU features; virtual shared variables and a logical thread mapping scheme hide the complex memory hierarchy and physical characteristics.…”