A GPU-based hyperbolic SVD algorithm

Novaković, Vedran; Singer, Sanja

doi:10.1007/s10543-011-0333-5

Cited by 30 publications

(25 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The eigenvalue order tends to stabilize eventually, thus no swapping is usually needed in the last few sweeps [28,29]. If the rotation is hyperbolic, to keep J partitioned, set V ′ pq = V pq (there is no sorting, and the new diagonal elements are not needed).…”

Section: A Single-gpu Algorithmmentioning

confidence: 99%

“…The new strategies may be seen as the generalizations of the Mantharam-Eberlein block-recursive strategy [27] to all even matrix orders. These new strategies are combined with the multilevel blocking and parallelization techniques explored in [20,21,37,36,29], to deliver the Jacobi-type (H)SVD algorithms for the graphics processing unit(s), competitive with the leading hybrid (CPU + GPU) alternatives, like MAGMA. The new algorithms are carefully designed to use a CPU primarily as a controlling unit.…”

Section: A Multi-gpu Algorithmmentioning

confidence: 99%

“…GPUs instead offer a complex memory hierarchy, with different access speeds and patterns, and both automatically and programmatically managed caches. Even more so than in the CPU world, a (less) careful hardware-adapted blocking of a GPU algorithm is the key technique by which considerable speedups are gained (or lost).After the introductory paper [29], here we present a family of the full block [21] and the block-oriented [20] one-sided Jacobi-type algorithm variants for the ordinary (SVD) and the hyperbolic singular value decomposition (HSVD) of a matrix, targeting both a single and the multiple GPUs. The blocking of our algorithm follows the levels of the GPU memory hierarchy; namely, the innermost level of blocking tries to maximize the amount of computation done inside the fastest (and smallest) memory of the registers and manual caches.…”

mentioning

confidence: 99%

“…After the introductory paper [29], here we present a family of the full block [21] and the block-oriented [20] one-sided Jacobi-type algorithm variants for the ordinary (SVD) and the hyperbolic singular value decomposition (HSVD) of a matrix, targeting both a single and the multiple GPUs. The blocking of our algorithm follows the levels of the GPU memory hierarchy; namely, the innermost level of blocking tries to maximize the amount of computation done inside the fastest (and smallest) memory of the registers and manual caches.…”

mentioning

confidence: 99%

See 3 more Smart Citations

A Hierarchically Blocked Jacobi SVD Algorithm for Single and Multiple Graphics Processing Units

Novaković¹

2015

SIAM J. Sci. Comput.

Self Cite

View full text Add to dashboard Cite

Abstract. We present a hierarchically blocked one-sided Jacobi algorithm for the singular value decomposition (SVD), targeting both single and multiple graphics processing units (GPUs). The blocking structure reflects the levels of GPU's memory hierarchy. The algorithm may outperform MAGMA's dgesvd, while retaining high relative accuracy. To this end, we developed a family of parallel pivot strategies on GPU's shared address space, but applicable also to inter-GPU communication. Unlike common hybrid approaches, our algorithm in a single GPU setting needs a CPU for the controlling purposes only, while utilizing GPU's resources to the fullest extent permitted by the hardware. When required by the problem size, the algorithm, in principle, scales to an arbitrary number of GPU nodes. The scalability is demonstrated by more than twofold speedup for sufficiently large matrices on a Tesla S2050 system with four GPUs vs. a single Fermi card. Key words. Jacobi (H)SVD, parallel pivot strategies, graphics processing units AMS subject classifications. 65Y05, 65Y10, 65F151. Introduction. Graphics processing units have become a widely accepted tool of parallel scientific computing, but many of the established algorithms still need to be redesigned with massive parallelism in mind. Instead of multiple CPU cores, which are fully capable of simultaneously processing different operations, GPUs are essentially limited to many concurrent instructions of the same kind-a paradigm known as SIMT (single-instruction, multiple-threads) parallelism.SIMT type of parallelism is not the only reason for the redesign. Modern CPU algorithms rely on (mostly automatic) multi-level cache management for speedup. GPUs instead offer a complex memory hierarchy, with different access speeds and patterns, and both automatically and programmatically managed caches. Even more so than in the CPU world, a (less) careful hardware-adapted blocking of a GPU algorithm is the key technique by which considerable speedups are gained (or lost).After the introductory paper [29], here we present a family of the full block [21] and the block-oriented [20] one-sided Jacobi-type algorithm variants for the ordinary (SVD) and the hyperbolic singular value decomposition (HSVD) of a matrix, targeting both a single and the multiple GPUs. The blocking of our algorithm follows the levels of the GPU memory hierarchy; namely, the innermost level of blocking tries to maximize the amount of computation done inside the fastest (and smallest) memory of the registers and manual caches. The GPU's global RAM and caches are considered by the mid-level, while inter-GPU communication and synchronization are among the issues addressed by the outermost level of blocking.At each blocking level an instance of either the block-oriented or the full block Jacobi (H)SVD is run, orthogonalizing pivot columns or block-columns by conceptually the same algorithm at the lower level. Thus, the overall structure of the algorithm is hierarchical (or recursive) in nature, and ready to fit not only the cur...

show abstract

Section: A Single-gpu Algorithmmentioning

confidence: 99%

Section: A Multi-gpu Algorithmmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

A Hierarchically Blocked Jacobi SVD Algorithm for Single and Multiple Graphics Processing Units

Novaković¹

2015

SIAM J. Sci. Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

“…It should be noted that an important drawback of hybrid algorithms is the need to copy data from the CPU to the GPU and vice versa, which leads to significant time delays [16], [18]. Communication costs are considerably reduced by means of filtration procedure.…”

Section: The Development Of Hybrid Cpu-gpu Algorithmmentioning

confidence: 99%

Innovative GPU accelerated algorithm for fast minimum convex hulls computation

Potebnia¹,

Pogorilyy²

2015

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

Abstract-Innovative algorithm for forming graph minimum convex hulls using the GPU is proposed. High speed and linear complexity of this method are achieved by distribution of the graph's vertices into separate units and their filtering. The key factor for improving the performance of innovative algorithm is the massively-parallel implementation of local hulls formation using video accelerators. A computational process is controlled by means of auxiliary matrices. A number of experimental studies of the algorithm have been carried out, and its suitability for application in the hull processing for large-scale problems has been demonstrated. The speed of the new method is 10 -20 times higher compared to using functions of the professional mathematical package Wolfram Mathematica.

show abstract

Divide‐and‐conquer approach for solving singular value decomposition based on MapReduce

Zhao

Tian

et al. 2014

Concurrency and Computation

View full text Add to dashboard Cite

Singular value decomposition (SVD) shows strong vitality in the area of information analysis and has significant application value in most of the scientific big data fields. However, with the rapid development of Internet, the information online reveals fast growing trend. For a large-scale matrix, applying SVD computation directly is both time consuming and memory demanding. There are many works available to speed up the computation of SVD based on the message passing interface model. However, to deal with largescale data processing, a MapReduce model has many advantages over a message passing interface model, such as fault tolerance, load balancing and simplicity. For a MapReduce environment, existing approaches only focus on low rank SVD approximation and tall-and-skinny matrix SVD computation, and there are no implementations of full rank SVD computation. In this paper, we propose a MapReduce-based implementation for solving divide-and-conquer SVD algorithm. To achieve high performance, we design a two-stage task scheduling strategy based on the mathematical characteristics of divide-and-conquer SVD algorithm. To further strengthen the performance, we propose a row-index-based divide algorithm, a pipelined task scheduling method, and revised block matrix multiplication in MapReduce framework. Experimental result shows the efficiency of our algorithm. Our implementation can accommodate full rank SVD computation of large-scale matrix very efficiently. 332 S. ZHAO ET AL. model. Hadoop relies on a distributed file system to share large-scale dataset among clusters. MapReduce procedure includes a shuffle phase where intermediate result is written on disk. Thus, we do not expect algorithms running in MapReduce framework to be faster than algorithms deployed in a state-of-the-art in-memory message passing interface(MPI) cluster. However, MapReduce model has many advantages over MPI model, which makes it attractive to process large-scale dataset. MapReduce model is designed to be fault tolerant while MPI model is not. Moreover, MapReduce model offers mechanisms to automatically deal with load balancing issues as well as input and output operations in a distributed environment, which greatly simplifies the programming. In today's information technology industry, many companies have employed Hadoop-distributed systems to deal with their daily business data.There are already implementations of SVD computation in MapReduce available. However, these implementations only focus on two aspects. The first one is low rank SVD approximation of matrices [3][4][5]. These research works are designed for those applications such as large-scale recommender systems in which only small fraction of singular values are needed. Nevertheless, for applications, such as matrix pseudo-inverse calculations that are widely used in large-scale medical and astronomy image processing field nowadays [6] and full rank principal component analysis in face recognition [7], performing a full rank SVD computation is required. Obviously, current implemen...

show abstract

A GPU-based hyperbolic SVD algorithm

Cited by 30 publications

References 21 publications

A Hierarchically Blocked Jacobi SVD Algorithm for Single and Multiple Graphics Processing Units

A Hierarchically Blocked Jacobi SVD Algorithm for Single and Multiple Graphics Processing Units

Innovative GPU accelerated algorithm for fast minimum convex hulls computation

Divide‐and‐conquer approach for solving singular value decomposition based on MapReduce

Contact Info

Product

Resources

About