DITVA: Dynamic Inter-Thread Vectorization Architecture

Abstract

International audienceIn the Single-Program Multiple-Data (SPMD) programming model, threads of an application exhibit very similar control flows and often execute the same instructions, but on different data. In this paper, we propose the Dynamic Inter-thread Vectorization Architecture (DITVA) to leverage the implicit Data Level Parallelism that exists across threads on SPMD applications. By assembling dynamic vector instructions at runtime, DITVA extends an in-order SMT processor with a dynamic inter-thread vector execution mode akin to the Single-Instruction, Multiple-Thread model of Graphics Processing Units. In this mode, multiple scalar threads running in lockstep share a single instruction stream and their respective instruction instances are aggregated into SIMD instructions. DITVA can leverage existing SIMD units and maintains binary compatibility with existing CPU architec-tures. To balance thread-and data-level parallelism, threads are statically grouped into fixed-size independently scheduled warps. Additionally, to maximize dynamic vector-ization opportunities, we adapt the fetch steering policy to favor thread synchronization within warps and thus improve lockstep execution. Our experimental evaluation of the DITVA architecture on the SPMD applications from the PARSEC and Rodinia OpenMP benchmarks show that a 4-warp × 4-lane 4-issue DITVA architecture with a realistic bank-interleaved cache achieves 1.55× higher performance compared to a 4-thread 4-issue SMT architecture with AVX instructions , while fetching and issuing 51% fewer instructions, and achieving an overall 24% energy reduction. DITVA also enables applications limited by memory to scale with higher bandwidth architectures. For instance, when the bandwidth is increased from 2GB/s to 16GB/s, we find that memory bound applications show an improvement in performance by 3× in comparison with the baseline SMT. Therefore, DITVA appears as a cost-effective design for achieving very high single-core performance on SPMD parallel sections

Simt-X

“…(2) a convergence detector to facilitate path re-convergence [Kalathingal et al 2017] ; (3) a Pathtable, which tracks path masks as the paths diverge and re-converge; (4) a Divergence Predictor for speculating warp divergences; and (5) a PIRAT for orchestrating SIMT-X renaming. SIMT-X has the capability to execute both statically vectorized SIMD instructions generated by the compiler, and instructions dynamically formed by the microarchitecture from scalar threads.…”

“…Considering general-purpose parallel applications, it is possible that MinSP-PC may cause livelocks [ElTantawy and Aamodt 2016]. Accordingly, a hybrid Round-Robin (RR) approach was proposed by Kalathingal et al [2017] to guarantee forward progress, activated periodically every (m + 1) × n cycles, where m is the number of threads/warp with a total of n warps.…”

“…SIMT-X adopts this RR/MinSP-PC policy, applying a conventional RR algorithm for interleaving and scheduling individual warps. The original RR/MinSP-PC policy [Kalathingal et al 2017], however, is used by the paths of a warp to guarantee forward progress and lockstep execution.…”

“…As previously mentioned, SIMT-X adopts path re-convergence detection using the RR/MinSP-PC policy implemented by Kalathingal et al [2017], however with PST hardware for OoO SIMT applicability. Instruction PCs are speculated in the front-end based on branch history and the first active scalar thread in a given path.…”

“…To support both static vector instructions and thread warp execution on the SIMD units, SIMT-X's register file is banked using the free banking method employed by the Dynamic Inter-Thread Vectorization Architecture (DITVA) architecture [Kalathingal et al 2017]. With the free banking method, a hash function Exclusive Or (XOR) is used to establish slice distribution across the lanes, providing a unique order per thread.…”