Implementing and Evaluating an Heterogeneous, Scalable, Tridiagonal Linear System Solver with OpenCL to Target FPGAs, GPUs, and CPUs

Abstract

Solving diagonally dominant tridiagonal linear systems is a common problem in scientific high-performance computing (HPC). Furthermore, it is becoming more commonplace for HPC platforms to utilise a heterogeneous combination of computing devices. Whilst it is desirable to design faster implementations of parallel linear system solvers, power consumption concerns are increasing in priority. This work presents the oclspkt routine. The oclspkt routine is a heterogeneous OpenCL implementation of the truncated SPIKE algorithm that can use FPGAs, GPUs, and CPUs to concurrently accelerate the solving of diagonally dominant tridiagonal linear systems. The routine is designed to solve tridiagonal systems of any size and can dynamically allocate optimised workloads to each accelerator in a heterogeneous environment depending on the accelerator’s compute performance. The truncated SPIKE FPGA solver is developed first for optimising OpenCL device kernel performance, global memory bandwidth, and interleaved host to device memory transactions. The FPGA OpenCL kernel code is then refactored and optimised to best exploit the underlying architecture of the CPU and GPU. An optimised TDMA OpenCL kernel is also developed to act as a serial baseline performance comparison for the parallel truncated SPIKE kernel since no FPGA tridiagonal solver capable of solving large tridiagonal systems was available at the time of development. The individual GPU, CPU, and FPGA solvers of the oclspkt routine are 110%, 150%, and 170% faster, respectively, than comparable device-optimised third-party solvers and applicable baselines. Assessing heterogeneous combinations of compute devices, the GPU + FPGA combination is found to have the best compute performance and the FPGA-only configuration is found to have the best overall estimated energy efficiency.

High Throughput Multidimensional Tridiagonal Systems Solvers on FPGAs

“…With the introduction of High-Level synthesis (HLS) tools, a number of more recent works [14], [15], [16], [29] implemented the Thomas, PCR, and Spike algorithms on FPGA using HLS tools. Many of these works did not demonstrate the solver working on full applications, with the exception of Lászl ó et al in 2015 [14] which compared a one factor Black-Scholes option pricing equation using explicit and implicit methods on different architectures such as multi core CPUs, GPUs, and FPGAs.…”

“…The FPGA performance with PCR is shown to be comparable to that of the GPU, but the Spike algorithm on the FPGA outperforms the GPU. Similarly Macintosh, et al in 2019 [15] uses OpenCL to develop oclspkt, a library that implements tridiagonal systems solvers targeting FPGAs, GPUs, and CPUs. oclspkt uses the truncated spike algorithm, and as such will not give exact solutions.…”

“…As such, our underlying goal is to understand the criteria for a given system solver to be amenable to FPGA implementation and uncover the limitations and profitability of such accelerators. Previous work on tridiagonal system solvers for FPGAs utilized both low-level hardware description languages [18], [28], [31] as well as high-level synthesis tools [3], [14], [15], [16], [29]. They demonstrated implementation of standard tridiagonal system solver algorithms (Thomas, PCR, and Spike), evaluating how to best utilize FPGA resources to maximize performance.…”

Vector Operations for Accelerating Expensive Bayesian Computations – A Tutorial Guide

“…OpenMP is also not the only way to access SIMD within C/C++. For example, OpenCL kernels may be compiled for CPUs that support SIMD units (Hurn et al, 2016;Macintosh et al, 2019). Instruction level intrinsic functions (available in R via the RcppXsimd package) allow advanced features such as efficient random variates for small vectors, but this approach is very challenging and akin to machine code.…”

FPGA Implementation of A∗ Algorithm for Real-Time Path Planning

“…e architecture was implemented in Xilinx Kintex-7 FPGA and compared to the software algorithm. FPGAs offer high flexibility to Application-Specific Integrated Circuit (ASIC) when implementing the algorithm with a high degree of parallelism [9,10]. Results show that 37-75 times performance enhancement could be achieved with the accelerator's clock frequency at 100 MHz.…”