Reducing the amount of out‐of‐core data access for GPU‐accelerated randomized SVD

Abstract

Summary We propose two acceleration methods, namely, Fused and Gram, for reducing out‐of‐core data access when performing randomized singular value decomposition (RSVD) on graphics processing units (GPUs). Out‐of‐core data here are data that are too large to fit into the GPU memory at once. Both methods accelerate GPU‐enabled RSVD using the following three schemes: (1) a highly tuned general matrix‐matrix multiplication (GEMM) scheme for processing out‐of‐core data on GPUs; (2) a data‐access reduction scheme based on one‐dimensional data partition; and (3) a first‐in, first‐out scheme that reduces CPU‐GPU data transfer using the reverse iteration. The Fused method further reduces the amount of out‐of‐core data access by merging two GEMM operations into a single operation. By contrast, the Gram method reduces both in‐core and out‐of‐core data access by explicitly forming the Gram matrix. According to our experimental results, the Fused and Gram methods improved the RSVD performance up to 1.7× and 5.2×, respectively, compared with a straightforward method that deploys schemes (1) and (2) on the GPU. In addition, we present a case study of deploying the Gram method for accelerating robust principal component analysis, a convex optimization problem in machine learning.

Working memory inspired hierarchical video decomposition with transformative representations

“…Randomized algorithms typically use nonuniform sampling to select a certain set of row and column vectors from the target matrix, which can achieve an important sampling selection with lower overhead and higher accuracy compared with that of the uniform sampling method. Coupled with large data matrix partition schemes and a partial (or truncated) SVD of a small matrix, randomized SVD algorithms can be implemented in parallel on graphics processing units (GPUs) with the capability of fast matrix multiplications and random number generations to achieve further acceleration [61], [62]. Nevertheless, the computational bottleneck restricting real-time performance still exists in the CPU-GPU transfer bandwidth and vector summation [61], [62] inherent in RPCA-based video decomposition.…”

“…Coupled with large data matrix partition schemes and a partial (or truncated) SVD of a small matrix, randomized SVD algorithms can be implemented in parallel on graphics processing units (GPUs) with the capability of fast matrix multiplications and random number generations to achieve further acceleration [61], [62]. Nevertheless, the computational bottleneck restricting real-time performance still exists in the CPU-GPU transfer bandwidth and vector summation [61], [62] inherent in RPCA-based video decomposition.…”

Integrating GPU support for FreeSurfer with OpenACC

“…GPUs are more powerful accelerator devices than manycore CPUs for computing-and memory-intensive applications [13]- [16]. CUDA [17] is a parallel computing platform based on C++ which can be used to access the instruction set and computational elements on Nvidia GPUs.…”