Learning-Based Application-Agnostic 3D NoC Design for Heterogeneous Manycore Systems

Abstract

The rising use of deep learning and other big-data algorithms has led to an increasing demand for hardware platforms that are computationally powerful, yet energy-efficient. Due to the amount of data parallelism in these algorithms, high-performance threedimensional (3D) manycore platforms that incorporate both CPUs and GPUs present a promising direction. However, as systems use heterogeneity (e.g., a combination of CPUs, GPUs, and accelerators) to improve performance and efficiency, it becomes more pertinent to address the distinct and likely conflicting communication requirements (e.g., CPU memory access latency or GPU network throughput) that arise from such heterogeneity. Unfortunately, it is difficult to quickly explore the hardware design space and choose appropriate tradeoffs between these heterogeneous requirements. To address these challenges, we propose the design of a 3D Networkon-Chip (NoC) for heterogeneous manycore platforms that considers the appropriate design objectives for a 3D heterogeneous system and explores various tradeoffs using an efficient machine learning (ML)-based multi-objective optimization (MOO) technique. The proposed design space exploration considers the various requirements of its heterogeneous components and generates a set of 3D NoC architectures that efficiently trades off these design objectives. Our findings show that by jointly considering these requirements (latency, throughput, temperature, and energy), we can achieve 9.6% better Energy-Delay Product on average at nearly iso-temperature conditions when compared to a thermally-optimized design for 3D heterogeneous NoCs. More importantly, our results suggest that our 3D NoCs optimized for a few applications can be generalized for unknown applications as well. Our results show that these generalized 3D NoCs only incur a 1.8% (36-tile system) and 1.1% (64-tile system) average performance loss compared to application-specific NoCs.

Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization

“…4) Network-On-Chip Design. Communication infrastructure is critical for efficient data movement in hardware chips (Joardar et al, 2018;Deshwal et al, 2019;Choi et al, 2018;Das et al, 2017) and they are designed using cycle-accurate simulators. We consider a dataset of 1024 configurations of a network-on-chip with ten input variables (d=10) (Che, Boyer, Meng, Tarjan, Sheaffer, Lee, & et al, 2009).…”

ReGraphX: NoC-enabled 3D Heterogeneous ReRAM Architecture for Training Graph Neural Networks

“…As a result, the communication backbone quickly becomes a performance bottleneck. In addition, the multi-hop nature of a planar-mesh NoC leads to higher communication latencies [13], which is not desirable for training GNNs. A 3D architecture can alleviate this problem and be a powerful enabler for the ReGraphX architecture.…”

“…A 3D architecture can alleviate this problem and be a powerful enabler for the ReGraphX architecture. By stacking multiple layers above each other, the physical distance between PE tiles is reduced significantly [13]. In addition, 3D NoCs enable high-performance multicast support [12]; both these characteristics are beneficial for GNN training.…”

“…In addition, 3D NoC enables high-throughput multicast support [12]. As a result, 3D NoC is more suited for long-range, multicast, and many-to-one-to-many traffic patterns [12], [13]. Hence, we conjecture that 3D NoC is a suitable communication backbone to enable the proposed ReRAMbased architecture for training GNNs.…”

Energy-Efficient Hybrid Unicore Architecture In Future Embedded Chip-Multiprocessor

“…Moreover, owing to the increasing computing capabilities, the connections between the GPUs and the CPUs are carried out by using off-chip interconnects that provide considerable traffic latency and high-power consumption [13][14][15]. A heterogeneous CMP that integrates CPUs and GPUs on the same chip with the sharing memory can solve this problem, avoid such expensive off-chip data transfers, and lead to improved system performance [16]. Moreover, the interconnection in a single chip should be more efficient and scalable to improve system performance.…”