An Open-Source Platform for High-Performance Non-Coherent On-Chip Communication

Abstract

On-chip communication infrastructure is a central component of modern systems-on-chip (SoCs), and it continues to gain importance as the number of cores, the heterogeneity of components, and the on-chip and off-chip bandwidth continue to grow. Decades of research on on-chip networks enabled cache-coherent shared-memory multiprocessors. However, communication fabrics that meet the needs of heterogeneous many-cores and accelerator-rich SoCs, which are not, or only partially, coherent, are a much less mature research area. In this work, we present a modular, topology-agnostic, high-performance on-chip communication platform. The platform includes components to build and link subnetworks with customizable bandwidth and concurrency properties and adheres to a state-of-the-art, industry-standard protocol. We discuss microarchitectural trade-offs and timing/area characteristics of our modules and show that they can be composed to build high-bandwidth (e.g., 2.5 GHz and 1024 bit data width) end-to-end on-chip communication fabrics (not only network switches but also DMA engines and memory controllers) with high degrees of concurrency. We design and implement a state-of-the-art ML training accelerator, where our communication fabric scales to 1024 cores on a die, providing 32 TB/s cross-sectional bandwidth at only 24 ns round-trip latency between any two cores.

Scale up your In-Memory Accelerator: Leveraging Wireless-on-Chip Communication for AIMC-based CNN Inference

“…In particular, we examine three aggregated interconnect bandwidths between CLs and L2 as wired: 22.4, 44.8, and 89.6 Gbit/s at f clock = 350 MHz, which corresponds to an interconnect bandwidth of 64, 128, and 256 bit/cycle, respectively. In this way, we span a wide range of available wired interconnect resources that can be instantiated in this kind of system [17]. Moreover, we assume a very optimistic latency of 9 cycles between CL and L2.…”

AGILER: An Adaptive Heterogeneous Tile-Based Many-Core Architecture for RISC-V Processors

“…Therefore, the growing number of PEs connected to the AXI interconnect leads to an increase in memory access latency per each PE and increases the probability of memory congestion due to limited crossbar bandwidth. In order to reduce memory congestion per tile, we use an open-source high-performance coherent AXI interconnect implementation [31], [32]. The AXI interconnect is based on a fully-connected crossbar where each slave port has a dedicated connection to each master port.…”

ManyGUI: A Graphical Tool to Accelerate Many-core Debugging Through Communication, Memory, and Energy Profiling

“…Some works adopt cache coherence protocols, as the distributed directory-based protocol [2,12]. Other works [8,9,15], argue that cache coherence protocols are not scalable for many-cores due to their high cost in terms of synchronization overhead and energy consumption observed, specifically, for streaming data-flow applications [15]. Instead, the alternative is to rely upon software-managed scratchpad memory close to each CPU, with the communication among CPUs initialized by software [1,7,8,14,15].…”