Semi-automatic restructuring of offloadable tasks for many-core accelerators

Abstract

Work division between the processor and accelerator is a common theme in modern heterogenous computing. Recent efforts (such as LEO and OpenAcc) provide directives that allow the developer to mark code regions in the original application from which offloadable tasks can be generated by the compiler. Auto-tuners and runtime schedulers work with the options (i.e., offloadable tasks) generated at compile time, which is limited by the directives specified by the developer. There is no provision for offload restructuring.We propose a new directive to add relaxed semantics to directive-based languages. The compiler identifies and generates one or more offloadable tasks in the neighbourhood of the code region marked by the directive. Central to our contribution is the idea of sub-offload and super-offload. In sub-offload, only a part of the code region marked by the developer is offloaded to the accelerator, while the other part executes on the CPU in parallel. This is done by splitting the index range of the main parallel loop into two or more parts and declaring one of the subloops as the offloadable task. Support is added to handle reduction variables and critical sections across subloops. Sub-offload enables concurrent execution of a task on the CPU and accelerator. In super-offload, a code region larger than the one specified by the developer is declared as the offloadable task (e.g., a parent loop). Super-offload reduces data transfers between CPU and accelerator memory.We develop Elastic Offload Compiler(EOC) for use alongside existing directive-based languages. The current implementation supports LEO for the new Intel Xeon Phi (MIC) architecture. We evaluate EOC with respect to SpecOMP and NAS Parallel Benchmarks. Speedups range between 1.3x-4.4x with the CPU version as baseline and 1.2x-24x with the offload (CPU-MIC) version as baseline.Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. SC '13,

Reverse Offload Programming on Heterogeneous Systems

“…In addition, numerous compilers and productivity tools have been developed to minimize programming effort and optimize performance for a MIC system. Ravi et al [31] propose a semi-automatic method for establishing the division of work between the CPU and MIC processors by providing new directives that added relaxed semantics to directive-based languages. Song et al [32] present source-to-source compiler optimizations (Comp) that can improve the performance of applications that offload computations to manycore processors.…”

An Investigation Into Partitioning Algorithms for Automatic Heterogeneous Compilers

“…Accelerating programs to potentially 40x of their normal speed is possible by taking advantage of heterogeneous parallelization. [2] Due to the silicon wall [3] with regards to transistor sizing [4], offloading [5] calculations onto heterogeneous hardware has also become an effective solution. Heterogeneous approaches are usually composed of a traditional processor combined with dedicated hardware logic.…”