Robust SIMD: Dynamically Adapted SIMD Width and Multi-Threading Depth

Abstract

Abstract-Architectures that aggressively exploit SIMD often have many datapaths execute in lockstep and use multithreading to hide latency. They can yield high throughput in terms of area-and energy-efficiency for many dataparallel applications. To balance productivity and performance, many recent SIMD organizations incorporate implicit cache hierarchies. Exaples of such architectures include Intel's MIC, AMD's Fusion, and NVIDIA's Fermi. However, unlike software-managed streaming memories used in conventional graphics processors (GPUs), hardware-managed caches are more disruptive to SIMD execution; therefore the interaction between implicit caching and aggressive SIMD execution may no longer follow the conventional wisdom gained from streaming memories. We show that due to more frequent memory latency divergence, lower latency in non-L1 data accesses, and relatively unpredictable L1 contention, cache hierarchies favor different SIMD widths and multi-threading depths than streaming memories. In fact, because the above effects are subject to runtime dynamics, a fixed combination of SIMD width and multi-threading depth no longer works ubiquitously across diverse applications or when cache capacities are reduced due to pollution or power saving.To address the above issues and reduce design risks, this paper proposes Robust SIMD, which provides wide SIMD and then dynamically adjusts SIMD width and multi-threading depth according to performance feedback. Robust SIMD can trade wider SIMD for deeper multi-threading by splitting a wider SIMD group into multiple narrower SIMD groups. Compared to the performance generated by running every benchmark on its individually preferred SIMD organization, the same Robust SIMD organization performs similarlysometimes even better due to phase adaptation-and outperforms the best fixed SIMD organization by 17%. When Dcache capacity is reduced due to runtime disruptiveness, Robust SIMD offers graceful performance degradation; with 25% polluted cache lines in a 32 KB D-cache, Robust SIMD performs 1.4× better compared to a conventional SIMD architecture.

Buddy SM

“…Most of these works require the reorganization of threads every cycle and introduce hardware overhead to support such reorganization. Several works studied the impact of SIMD width on performance [Meng et al 2012;Lashgar et al 2012] and concluded that large warps improve the memory access coalescing but suffer more synchronization overhead, memory divergences, and branch divergences. Their evaluations showed that most benchmarks exhibited performance degradation when running large warps where NK-threaded warps running on K-wide SIMD lanes.…”

“…As shown in Figure 9, the Fermi GPU chip has roughly 23mm × 23mm die size and is manufactured in 40nm technology [Valich 2010], so we estimate that the communication stage takes one-cycle latency to traverse the distance between two adjacent SMs, which is roughly 5mm long. The delay of 5mm wire is 0.3ns (3.3GHz) reported by CACTI 6.0 [Muralimanohar et al 2009]. We clock the NoC at 1.4GHz, which not only meets one cycle latency but also provides headroom for low-swing voltage operation that causes 0.6ns wire delay and 0.1ns signal regeneration that still meets timing.…”

Pannotia: Understanding irregular GPGPU graph applications

“…Similarly, long wavefronts can be time-sliced to execute on these narrow SIMD units or wide SIMDs as usual. Prior research has also investigated using a dynamic approach adapting SIMD width [30].…”

Maximizing Energy Efficiency through Vertically Integrated Dynamic Reconfigurability

“…However, practically speaking, only few of those components are executing code at a time; and depending on the benchmark, some components are exercised more than others. To better estimate the energy consumption breakdown in the processor during typical benchmarks, we used the architectural simulator gem5 [35] to count the number of instructions that triggers the different components during the execution of several benchmarks used before in [37]. Those benchmarks are extracted from MineBench [38], SPLASH-2 [39], and Rodinia [40].…”