FPGA implementation of a configurable cache/scratchpad memory with virtualized user-level RDMA capability

Kalokerinos, George; Papaefstathiou, Vassilis; Nikiforos, George; Kavadias, Stamatis; Katevenis, Manolis; Pnevmatikatos, Dionisios; Yang, Xiaojun

doi:10.1109/icsamos.2009.5289226

Cited by 20 publications

(14 citation statements)

References 25 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to optimize our cache-integrated design at a lower level, we carefully adapt the format of transfer descriptions via command buffers to the NoC packet header format, so that we can exploit corresponding field alignment. This approach saves about 19.3% of the logic in our integrated design, over our measurement reported in [15]. In the results presented, we assume that 1 FPGA lookup table (LUT) or 1 flip-flop is equivalent to 8 gates.…”

Section: Logic Cost Of Cache-integrated Ni Mechanismsmentioning

confidence: 83%

“…3 in a hardware prototype based on a Xilinx Virtex-5 FPGA. A previous version of the prototype was presented in [14]. The current version is a major rewrite of the code, carefully pursuing logic reuse, implementing event responses, three levels of NoC priority and some other features not present in the version of [14].…”

Section: The Hardware Prototypementioning

confidence: 99%

“…A previous version of the prototype was presented in [14]. The current version is a major rewrite of the code, carefully pursuing logic reuse, implementing event responses, three levels of NoC priority and some other features not present in the version of [14]. The current version of the prototype was presented in [15], and we report here on the reduced logic cost, that resulted from some alignment optimizations in that version.…”

Section: The Hardware Prototypementioning

confidence: 99%

See 2 more Smart Citations

Cache-Integrated Network Interfaces: Flexible On-Chip Communication and Synchronization for Large-Scale CMPs

Kavadias

Katevenis

Zampetakis

et al. 2011

Int J Parallel Prog

Self Cite

View full text Add to dashboard Cite

Per-core scratchpad memories (or local stores) allow direct inter-core communication, with latency and energy advantages over coherent cache-based communication, especially as CMP architectures become more distributed. We have designed cache-integrated network interfaces, appropriate for scalable multicores, that combine the best of two worlds -the flexibility of caches and the efficiency of scratchpad memories: on-chip SRAM is configurably shared among caching, scratchpad, and virtualized network interface (NI) functions. This paper presents our architecture, which provides local and remote scratchpad access, to either individual words or multiword blocks through RDMA copy. Furthermore, we introduce event responses, as a technique that enables software configurable communication and synchronization primitives. We present three event response mechanisms that expose NI functionality to software, for multiword transfer initiation, completion notifications for software selected sets of arbitrary size transfers, and multi-party synchronization queues. We implemented these mechanisms in a four-core FPGA prototype, and measure the logic overhead over a cache-only design for basic NI functionality to be less than 20%. We also evaluate the on-chip communication performance on the prototype, as well as the 123Int J Parallel Prog performance of synchronization functions with simulation of CMPs with up to 128 cores. We demonstrate efficient synchronization, low-overhead communication, and amortized-overhead bulk transfers, which allow parallelization gains for fine-grain tasks, and efficient exploitation of the hardware bandwidth.

show abstract

Section: Logic Cost Of Cache-integrated Ni Mechanismsmentioning

confidence: 83%

Section: The Hardware Prototypementioning

confidence: 99%

Section: The Hardware Prototypementioning

confidence: 99%

See 1 more Smart Citation

Cache-Integrated Network Interfaces: Flexible On-Chip Communication and Synchronization for Large-Scale CMPs

Kavadias

Katevenis

Zampetakis

et al. 2011

Int J Parallel Prog

Self Cite

View full text Add to dashboard Cite

show abstract

“…We present the design and implementation of an OpenMP runtime system for an FPGA prototype of the SARC architecture [9] which features explicitly managed on-chip local memories, explicit on-chip communication primitives including remote stores for producer-initiated short data transfers, RDMA operations for producer-initiated or consumer-initiated bulk data transfers, hardware event queues with automatically generated responses, and hardware counters [4].…”

Section: Introductionmentioning

confidence: 99%

Fine-grain OpenMP runtime support with explicit communication hardware primitives

Tendulkar

Papaefstathiou

Nikiforos

et al. 2011

2011 Design, Automation &Amp; Test in Europe

View full text Add to dashboard Cite

Abstract-We present a runtime system that uses the explicit on-chip communication mechanisms of the SARC multi-core architecture, to implement efficiently the OpenMP programming model and enable the exploitation of fine-grain parallelism in OpenMP programs. We explore the design space of implementation of OpenMP directives and runtime intrinsics, using a family of hardware primitives; remote stores, remote DMAs, hardware counters and hardware event queues with automatic responses, to support static and dynamic scheduling and data transfers in local memories. Using an FPGA prototype with four cores, we achieve OpenMP task creation latencies of 30-35 processor clock cycles, initiation of parallel contexts in 50 cycles and synchronization primitives in 65-210 cycles.

show abstract

“…Our work on explicit communication and synchronization for the SARC architecture includes an FPGA prototype described in [2] and a longer description of the architecture, with performance measurements collected on the FPGA prototype [13].…”

Section: Related Workmentioning

confidence: 99%

Explicit Communication and Synchronization in SARC

et al. 2010

Self Cite

View full text Add to dashboard Cite

SARC merges cache controller and network interface functions by relying on a single hardware primitive: each access checks the tag and the state of the addressed line for possible occurrence of events that may trigger responses like coherence actions, RDMA, synchronization, or configurable event notifications. The fully virtualized and protected user-level API is based on specially marked lines in the scratchpad space that respond as command buffers, counters, or queues. The runtime system maps communication abstractions of the programming model to data transfers among local memories using remote write or read DMA and into task synchronization and scheduling using notifications, counters, and queues. The on-chip network provides efficient communication among these configurable memories, using advanced topologies and routing algorithms, and providing for process variability in NoC links. We simulate benchmark kernels on a full-system simulator to compare speedup and network traffic against cache-only systems with directory-based coherence and prefetchers. Explicit communication provides 10 to 40% higher speedup on 64 cores, and reduces network traffic by factors of 2 to 4, thus economizing on energy and power; lock and barrier latency is reduced by factors of 3 to 5. EXPLICIT COMMUNICATION AND NETWORK INTERFACE EVOLUTIONInterprocessor communication (IPC) is the basis of parallel processing. IPC can be implicit, when the addresses supplied by the software do not identify physical data locations or (time of) movement, or it can be explicit, when software (the application, or compiler, or runtime system) is able to also indicate physical placement or transfers, besides specifying computation on data. The SARC architecture [1], supports both implicit IPC, through cache coherence, for ease of programming, and explicit IPC, through scratchpad memories and remote store instructions or remote DMA operations, to be used by software whenever possible for achieving scalable performance.In order to hide IPC latency, when using implicit communication, we need large issue windows in out-of-order-execution processors, or sophisticated data prefetchers, or both. Explicit communication has the potential to better hide IPC latency, in those cases when software knows better than hardware what transfers need to take place and when. Remote store instructions, to addresses that indicate proximity to the consumer, when that is known at production time, will transfer data at the earliest possible time; hardware should coalesce writes to adjacent targets into few network packets, and the processor should not wait for the arrival acknowledgments. Remote direct memory access (RDMA) is the other method for explicit communication, in cases that require either reads -when the consumer is unknown or unavailable at production time-or multi-word writes -to achieve good coalescence.Traditional systems viewed networks as external (slow) devices, provided DMA in the network interface (NI), and interacted to it through (slow) input/output (I...

show abstract

FPGA implementation of a configurable cache/scratchpad memory with virtualized user-level RDMA capability

Cited by 20 publications

References 25 publications

Cache-Integrated Network Interfaces: Flexible On-Chip Communication and Synchronization for Large-Scale CMPs

Cache-Integrated Network Interfaces: Flexible On-Chip Communication and Synchronization for Large-Scale CMPs

Fine-grain OpenMP runtime support with explicit communication hardware primitives

Explicit Communication and Synchronization in SARC

Contact Info

Product

Resources

About