Benefits of Adding Hardware Support for Broadcast and Reduce Operations in MPSoC Applications

Abstract

MPI has been used as a parallel programming model for supercomputers and clusters and recently in MultiProcessor Systems-on-Chip (MPSoC). One component of MPI is collective communication and its performance is key for certain parallel applications to achieve good speedups. Previous work showed that, with synthetic communication-only benchmarks, communication improvements of up to 11.4-fold and 22-fold for broadcast and reduce operations, respectively, can be achieved by providing hardware support at the network level in a Network-on-Chip (NoC). However, these numbers do not provide a good estimation of the advantage for actual applications, as there are other factors that affect performance besides communications, such as computation. To this end, we extend our previous work by evaluating the impact of hardware support over a set of five parallel application kernels of varying computation-to-communication ratios. By introducing some useful computation to the performance evaluation, we obtain more representative results of the benefits of adding hardware support for broadcast and reduce operations. The experiments show that applications with lower computation-to-communication ratios benefit the most from hardware support as they highly depend on efficient collective communications to achieve better scalability. We also extend our work by doing more analysis on clock frequency, resource usage, power, and energy. The results show reasonable scalability for resource utilization and power in the network interfaces as the number of channels increases and that, even though more power is dissipated in the network interfaces due to the added hardware, the total energy used can still be less if the actual speedup is sufficient. The application kernels are executed in a 24-embedded-processor system distributed across four FPGAs.

High Performance Communication on Reconfigurable Clusters

“…A major application was Molecular Dynamic simulations. Arun et al instantiated soft processors on each FPGA and implemented a specialized MPI library called TMD-MPI to handle inter-FPGA communication (Peng et al, 2014). While soft cores have the potential for generality, they can cause high latency on critical paths.…”

“…TMD-MPI (Saldana and Chow, 2006) is a software-based communication library that facilitates inter-FPGA communication for TMD system. They released an improved version in 2014 (Peng et al, 2014) by offloading MPI Bcast and Reduce into FPGA logic. Andrew, et al made a similar effort in (Schmidt et al, 2012).…”

“…There have been some attempts to make progress on this topic, but little of that work has been systematic. Mostly it has relied either on the use of softcore processors (Peng et al, 2014;Schmidt et al, 2012) or on latency-insensitive (LI) channels (Fleming et al, 2012). These approaches achieve generality, but at the cost of significant performance degradation.…”

An efficient FPGA implementation of QR decomposition using a novel systolic array architecture based on enhanced vectoring CORDIC

“…The difference is the estimated power of the circuit of interest. This approach is described in more detail in Peng et al [17]. Since the power estimation is a function of the toggle rate of the signals, a range of toggle rates is used to observe the sensitivity of the power to the toggle rate.…”