Explicit Communication and Synchronization in SARC

Abstract

SARC merges cache controller and network interface functions by relying on a single hardware primitive: each access checks the tag and the state of the addressed line for possible occurrence of events that may trigger responses like coherence actions, RDMA, synchronization, or configurable event notifications. The fully virtualized and protected user-level API is based on specially marked lines in the scratchpad space that respond as command buffers, counters, or queues. The runtime system maps communication abstractions of the programming model to data transfers among local memories using remote write or read DMA and into task synchronization and scheduling using notifications, counters, and queues. The on-chip network provides efficient communication among these configurable memories, using advanced topologies and routing algorithms, and providing for process variability in NoC links. We simulate benchmark kernels on a full-system simulator to compare speedup and network traffic against cache-only systems with directory-based coherence and prefetchers. Explicit communication provides 10 to 40% higher speedup on 64 cores, and reduces network traffic by factors of 2 to 4, thus economizing on energy and power; lock and barrier latency is reduced by factors of 3 to 5. EXPLICIT COMMUNICATION AND NETWORK INTERFACE EVOLUTIONInterprocessor communication (IPC) is the basis of parallel processing. IPC can be implicit, when the addresses supplied by the software do not identify physical data locations or (time of) movement, or it can be explicit, when software (the application, or compiler, or runtime system) is able to also indicate physical placement or transfers, besides specifying computation on data. The SARC architecture [1], supports both implicit IPC, through cache coherence, for ease of programming, and explicit IPC, through scratchpad memories and remote store instructions or remote DMA operations, to be used by software whenever possible for achieving scalable performance.In order to hide IPC latency, when using implicit communication, we need large issue windows in out-of-order-execution processors, or sophisticated data prefetchers, or both. Explicit communication has the potential to better hide IPC latency, in those cases when software knows better than hardware what transfers need to take place and when. Remote store instructions, to addresses that indicate proximity to the consumer, when that is known at production time, will transfer data at the earliest possible time; hardware should coalesce writes to adjacent targets into few network packets, and the processor should not wait for the arrival acknowledgments. Remote direct memory access (RDMA) is the other method for explicit communication, in cases that require either reads -when the consumer is unknown or unavailable at production time-or multi-word writes -to achieve good coalescence.Traditional systems viewed networks as external (slow) devices, provided DMA in the network interface (NI), and interacted to it through (slow) input/output (I...

Design space exploration for fair resource-allocated NoC architectures

“…Both types of packets also include some protocol/network headers. The r c\d ratio is used to evaluate the control relative to data traffic volume, similar to [11,17]. This work considers two corner cases, more control packets per data packet (r c\d > 1) and more data packets per control packet (r c\d < 1).…”

NP-SARC: Scalable network processing in the SARC multi-core FPGA platform

“…On the other hand, using a shared queue results to higher communication traffic, since the packets have first to travel to the shared queue and then to the second stage (thus the shared queue results in higher power consumption). A detailed performance evaluation against other schemes such as directory-based cache coherence with or without hardware prefetching has been presented in [27].…”

Formic: Cost-efficient and Scalable Prototyping of Manycore Architectures

“…The first hardware design project which uses the Formic board is a prototype of a non cache-coherent manycore architecture, based on ideas of the SARC project [3], which was fully implemented in software simulation and partially implemented on a XUPV5 hardware platform [4]. Each board fits in its FPGA eight CPUs, their private L1 and L2 caches, eight GTP links and a full network-on-chip centered around a 22-port crossbar.…”

“…The Counter & Mailbox (CMX) block keeps 128 counters that can be used to track the progress of ongoing DMA operations [3]. The counters can be polled by the CPU, send an interrupt and/or send notification packets to other counters when the programmed number of acknowledgment packets has been received.…”