Singular value decomposition utilizing parallel algorithms on graphical processors

Batched QR and SVD algorithms on GPUs with applications in hierarchical matrix compression

“…GPU routines optimized for computing the QR decomposition of very tall and skinny matrices are presented in [10] where they develop an efficient transpose matrix-vector computation that is employed with some minor changes in this work. GPU-CPU hybrid algorithms for batched SVD using Jacobi and bidiagonalization methods are introduced in [11] where pair generation for the Jacobi method and the solver phase of the bidiagonalization are handled on the CPU. The work in [12] employs the power method to construct a rank 1 approximation for 2D filters in convolutional neural networks.…”

Using reconfigurable computing technology to accelerate matrix decomposition and applications

“…However, its inherent high data dependency poses challenges for further parallelism. The GPU-based implementations for the Householder approach were evaluated, in which possible acceleration was demonstrated only for matrices with significantly large sizes ( 1000 × 1000) [Lahabar and Narayanan (2009); Kotas and Barhen (2011)].…”

“…The Hestenes-Jacobi Method [Strumpen et al (2003)], which is also known as one-sided Jacobi rotation, provides a better opportunity for vectorized parallel operations. However, its architectural design with iterative and repetitive processing limited the overall speedup [Kotas and Barhen (2011)…”

“…The dimensional speedups that can be achieved range from 3.8× to 43.6× for matrices with column sizes from 128 to 256 and row dimensions from 128 to 2048. FPGA-based design [Kotas and Barhen (2011)] was devised to perform fixed-point operations, which can only analyze the matrices with the size up to 32 × 128 due to the limitation of on-chip memory. Although the better performance has been demonstrated compared to their software execution with Matlab SVD for matrices with dimensions range from 2 × 2 to 32 × 127, our Matlab SVD routine runs 100 times faster than their Matlab SVD, and shares comparable speeds with their FPGA-based design.…”

“…The recent emergence of Graphic Processing Units (GPUs) in the high performance computing community has allowed for new methods to accelerate many general-purpose computations. However, the multi-dimensional threading structure of GPU computing is not highly compatible with the iterative thread synchronization and irregular memory access required for better EVD convergence making the optimization of these designs on GPUs quite challenging, especially for input matrices with dimensions smaller than 1000 [Lahabar and Narayanan (2009); Kotas and Barhen (2011)].…”