Zedwulf: Power-Performance Tradeoffs of a 32-Node Zynq SoC Cluster

Moorthy, Pradeep; Kapre, Nachiket

doi:10.1109/fccm.2015.37

Cited by 34 publications

(14 citation statements)

References 15 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zedwulf is a 32-node multi-FPGA SoC. The inter-FPGA communication is implemented by connecting all the 32 FPGAs via an Ethernet switch (Moorthy and Kapre, 2015). Although it is a good practice at multi-FPGA SoC implementation, the Ethernet-based interconnection introduces much higher latency overhead than MGT interconnection.…”

Section: Zedwulfmentioning

confidence: 99%

See 1 more Smart Citation

High Performance Communication on Reconfigurable Clusters

Sheng

Yang

Herbordt

2018

2018 28th International Conference on Field Programmable Logic and Applications (FPL)

View full text Add to dashboard Cite

High Performance Computing (HPC) has matured to where it is an essential third pillar, along with theory and experiment, in most domains of science and engineering. Communication latency is a key factor that is limiting the performance of HPC, but can be addressed by integrating communication into accelerators. This integration allows accelerators to communicate with each other without CPU interactions, and even bypassing the network stack. Field Programmable Gate Arrays (FPGAs) are the accelerators that currently best integrate communication with computation. The large number of Multi-gigabit Transceivers (MGTs) on most high-end FPGAs can provide high-bandwidth and low-latency inter-FPGA connections. Additionally, the reconfigurable FPGA fabric enables tight coupling between computation kernel and network interface. Our thesis is that an application-aware communication infrastructure for a multi-FPGA system makes substantial progress in solving the HPC communication bottleneck. This dissertation aims to provide an application-aware solution for communication infrastructure for FPGA-centric clusters. Specifically, our solution demonstrates application-awareness across multiple levels in the network stack, including low-level vii link protocols, router microarchitectures, routing algorithms, and applications. We start by investigating the low-level link protocol and the impact of its latency variance on performance. Our results demonstrate that, although some link jitter is always present, we can still assume near-synchronous communication on an FPGAcluster. This provides the necessary condition for statically-scheduled routing. We then propose two novel router microarchitectures for two different kinds of workloads: a wormhole Virtual Channel (VC)-based router for workloads with dynamic communication, and a statically-scheduled Virtual Output Queueing (VOQ)-based router for workloads with static communication. For the first (VC-based) router, we propose a framework that generates application-aware router configurations. Our results show that, by adding application-awareness into router configuration, the network performance of FPGA clusters can be substantially improved. For the second (VOQbased) router, we propose a novel offline collective routing algorithm. This shows a significant advantage over a state-of-the-art collective routing algorithm. We apply our communication infrastructure to a critical strong-scaling HPC kernel, the 3D FFT. The experimental results demonstrate that the performance of our design is faster than that on CPUs and GPUs by at least one order of magnitude (achieving strong scaling for the target applications). Surprisingly, the FPGA cluster performance is similar to that of an ASIC-cluster. We also implement the 3D FFT on another multi-FPGA platform: the Microsoft Catapult II cloud. Its performance is also comparable or superior to CPU and GPU HPC clusters. The second application we investigate is Molecular Dynamics Simulation (MD). We model MD on both FPGA clouds and clusters. We find ...

show abstract

Section: Zedwulfmentioning

confidence: 99%

“…The back-end network with MGT links has no router support. In Zedwulf, FPGAs communicate purely through commodity Ethernet (Moorthy and Kapre, 2015). Few details are publicly available about the Catapult I router (Putnam et al, 2014).…”

Section: Previous Work Of Wormhole Vc-based Routers On Fpgasmentioning

confidence: 99%

High Performance Communication on Reconfigurable Clusters

Sheng

Yang

Herbordt

2018

2018 28th International Conference on Field Programmable Logic and Applications (FPL)

View full text Add to dashboard Cite

show abstract

“…Hence, there is an inherent need to reduce the time spent in fulfilling the network operations for maximizing performance gains while using distributed systems. We designed an optimized graph-oriented global scatter technique [8] using the Message Passing Interface (MPI) library. Step 1 and…”

Section: Mpi Optimizationmentioning

confidence: 99%

A Case for Embedded FPGA-based SoCs in Energy-Efficient Acceleration of Graph Problems

Kapre

Moorthy

2015

JSFI

View full text Add to dashboard Cite

Sparse graph problems are notoriously hard to accelerate on conventional platforms due to irregular memory access patterns resulting in underutilization of memory bandwidth. These bottlenecks on traditional x86-based systems mean that sparse graph problems scale very poorly, both in terms of performance and power efficiency. A cluster of embedded SoCs (systems-on-chip) with closely-coupled FPGA accelerators can support distributed memory access with better matched low-power processing. We first conduct preliminary experiments across a range of COTS (commercial off-the-shelf) embedded SoCs to establish promise for energy-efficiency acceleration of sparse problems. We select the Xilinx Zynq SoC with FPGA accelerators to construct a prototype 32-node Beowulf cluster. We develop specialized MPI routines and memory DMA offload engines to support irregular communication efficiently. In this setup, we use the ARM processor as a data marshaller for local DMA traffic as well as remote MPI traffic while the FPGA may be used as a programmable accelerator. Across a set of benchmark graphs, we show that 32-node embedded SoC cluster can exceed the energy efficiency of an Intel E5-2407 by as much as 1.7× at a total graph processing capacity of 91-95 MTEPS for graphs as large as 32 million nodes and edges.

show abstract

“…For example, ARMbased SoCs with Field-Programmable Gate Array (FPGA) logic have shown significant advantages in power consumption and efficiency in executing highly parallel computation [14]. Other examples of the efficient use of heterogeneous SoCs in server setups are the ZCluster [18] and Zedwulf [19], which improves performance over similar clusters based solely on either ARM CPUs or FPGA enabling elasticity to trade power with performance. Compared to commodity servers, these SoC solutions improve energy-efficiency out of the box.…”

Section: Introductionmentioning

confidence: 99%

“…based on Hadoop [18]. Zedwulf is another SoC cluster which allows exploration of the performance-power trade-off for sparse graph applications [19].…”

Section: Introductionmentioning

confidence: 99%