Dynamic warp subdivision for integrated branch and memory divergence tolerance

Abstract

SIMD organizations amortize the area and power of fetch, decode, and issue logic across multiple processing units in order to maximize throughput for a given area and power budget. However, throughput is reduced when a set of threads operating in lockstep (a warp) are stalled due to long latency memory accesses. The resulting idle cycles are extremely costly. Multi-threading can hide latencies by interleaving the execution of multiple warps, but deep multi-threading using many warps dramatically increases the cost of the register files (multi-threading depth × SIMD width), and cache contention can make performance worse. Instead, intra-warp latency hiding should first be exploited. This allows threads that are ready but stalled by SIMD restrictions to use these idle cycles and reduces the need for multi-threading among warps. This paper introduces dynamic warp subdivision (DWS), which allows a single warp to occupy more than one slot in the scheduler without requiring extra register file space. Independent scheduling entities allow divergent branch paths to interleave their execution, and allow threads that hit to run ahead. The result is improved latency hiding and memory level parallelism (MLP). We evaluate the technique on a coherent cache hierarchy with private L1 caches and a shared L2 cache. With an area overhead of less than 1%, experiments with eight data-parallel benchmarks show our technique improves performance on average by 1.7X.

Dynamic Inter-Thread Vectorization Architecture: Extracting DLP from TLP

“…For instance, more flexibility could be obtained using Dynamic Warp Formation [24] or Simultaneous Branch Interweaving [25], Dynamic Warp Subdivision [9] could improve latency tolerance by allowing threads to diverge on partial cache misses, and Dynamic Scalarization [29] could further unify redundant dataflow across threads.…”

“…The concept of IS corresponds to warp-split [9] in the GPU architecture literature. While thread-to-warp assignment is static, a threadto-IS assignment is dynamic: the number of IS per warp may vary from 1 to m during execution, as does the number of threads per IS.…”

Reconvergence de contrôle implicite pour les architectures SIMT

“…Contrairement à la proposition de Takahashi, elle ne nécessite pas d'exécuter des blocs de base plus souvent que nécessaire. Meng, Tarjan et Skadron considèrent dans le cadre de leur technique de subdivision dynamique de warps une reconvergence opportuniste lorsque les PC de plusieurs threads coïncident (Meng et al, 2010). Cependant, le principal moyen de reconvergence repose toujours sur des points de synchronisation explicites et une pile de masques.…”

“…Une fois ces données arrivées, un arbitrage de PC a lieu pour donner l'opportunité à ces threads de « rattraper » les autres. Ce mécanisme permet aux threads ayant réussi leur accès au cache de prendre de l'avance sur ceux qui sont bloqués sur un échec, offrant gratuitement une fonctionnalité analogue à la subdivision dynamique de warps (Meng et al, 2010). L'architecture proposée est présentée figure 5a.…”

“…Les techniques de gestion de la divergence et de la reconvergence forment la base indispensable sur laquelle peuvent s'appuyer des politiques d'ordonnancement de threads de plus haut niveau tels que la formation dynamique de warps (W. W. L. Fung et al, 2009), la subdivision dynamique de warps (Meng et al, 2010) ou l'entrelacement simultané de branches (Brunie et al, 2012). Ces techniques complémentaires pourront bénéficier des avancées dans le domaine de la gestion de la reconvergence.…”