CD-Xbar: A Converge-Diverge Crossbar Network for High-Performance GPUs

Abstract

Modern GPUs feature an increasing number of streaming multiprocessors (SMs) to boost system throughput. How to construct an efficient and scalable network-on-chip (NoC) for future high-performance GPUs is particularly critical. Although a mesh network is a widely used NoC topology in manycore CPUs for scalability and simplicity reasons, it is ill-suited to GPUs because of the many-to-few-to-many traffic pattern observed in GPU-compute workloads. Although a crossbar NoC is a natural fit, it does not scale to large SM counts while operating at high frequency. In this paper, we propose the converge-diverge crossbar (CD-Xbar) network with round-robin routing and topology-aware concurrent thread array (CTA) scheduling. CD-Xbar consists of two types of crossbars, a local crossbar and a global crossbar. A local crossbar converges input ports from the SMs into so-called converged ports; the global crossbar diverges these converged ports to the last-level cache (LLC) slices and memory controllers. CD-Xbar provides routing path diversity through the converged ports. Round-robin routing and topology-aware CTA scheduling balance network traffic among the converged ports within a local crossbar and across crossbars, respectively. Compared to a mesh with the same bisection bandwidth, CD-Xbar reduces NoC active silicon area and power consumption by 52.5% and 48.5%, respectively, while at the same time improving performance by 13.9% on average. CD-Xbar performs within 2.9% of an idealized fully-connected crossbar. We further demonstrate CD-Xbar's scalability, flexibility and improved performance per Watt (by 17.1%) over state-of-the-art GPU NoCs which are highly customized and non-scalable. Index Terms-graphics processing unit (GPU), network-on-chip (NoC), crossbar !

Selective Replication in Memory-Side GPU Caches

“…The SM-and MC-routers have a 4-stage pipeline. We adopt two-level round-robin (2L-RR) CTA scheduling to balance the workload across the different SM-routers, i.e., 2L-RR first distributes CTAs across SM-routers and then across SMs within an SM-router [74], [75]. Other CTA scheduling policies are explored in the sensitivity analysis.…”

“…More specifically, the hardware cost of the full crossbar increases quadratically with the number of ports. Prior work [74], [75] observed that a hierarchical crossbar (H-Xbar) scales better than a full crossbar in terms of hardware complexity and power efficiency while providing the same bisection bandwidth. The H-Xbar achieves this through a twolevel router structure, i.e., the SM-router and the MC-router in Figure 1, in which each router is shared by clusters of SMs and LLC slices, respectively.…”