Orthogonalization on a General Purpose Graphics Processing Unit with Double Double and Quad Double Arithmetic

Abstract

Our problem is to accurately solve linear systems on a general purpose graphics processing unit with double double and quad double arithmetic. The linear systems originate from the application of Newton's method on polynomial systems. Newton's method is applied as a corrector in a path tracking method, so the linear systems are solved in sequence and not simultaneously. One solution path may require the solution of thousands of linear systems. In previous work we reported good speedups with our implementation to evaluate and differentiate polynomial systems on the NVIDIA Tesla C2050. Although the cost of evaluation and differentiation often dominates the cost of linear system solving in Newton's method, because of the limited bandwidth of the communication between CPU and GPU, we cannot afford to send the linear system to the CPU for solving during path tracking.Because of large degrees, the Jacobian matrix may contain extreme values, requiring extended precision, leading to a significant overhead. This overhead of multiprecision arithmetic is our main motivation to develop a massively parallel algorithm. To allow overdetermined linear systems we solve linear systems in the least squares sense, computing the QR decomposition of the matrix by the modified Gram-Schmidt algorithm. We describe our implementation of the modified Gram-Schmidt orthogonalization method using double double and quad double arithmetic for GPUs. Our experimental results on the NVIDIA Tesla C2050 and K20C show that the achieved speedups are sufficiently high to compensate for the overhead of one extra level of precision.Keywords double double arithmetic, general purpose graphics processing unit (GPU), massively parallel algorithm, modified Gram-Schmidt method, orthogonalization, quad double arithmetic, quality up.

Accelerating polynomial homotopy continuation on a graphics processing unit with double double and quad double arithmetic

“…In [43], we continued this line of investigation on the GPU, based on our GPU implementations of evaluation and differentiation algorithms [40], combining our GPU implementation of the the Gram-Schmidt orthogonalization method [41]. The computational results in [40] and [41] were on randomly generated data. The data in this paper comes from relevant polynomial systems, relevant to actual applications.…”

“…In particular, the cyclic n-root problems relate to the construction of complex Hadamard matrices [36] and the Pieri homotopies solve the output placement problem in linear systems control [10]. This paper reports on improvements and the integration of our building blocks (described in [40,41,43]) in an accelerated path tracker. Good speedups relative to the CPU are obtained on benchmark problems, sufficiently large enough to compensate for the computational overhead caused by the double double arithmetic.…”

“…In [39], we experimentally showed that the cost overhead of double double arithmetic is of a similar magnitude as the cost of complex double arithmetic and that eight CPU cores may suffice to offset this overhead in multithreaded implementations. In [43], we continued this line of investigation on the GPU, based on our GPU implementations of evaluation and differentiation algorithms [40], combining our GPU implementation of the the Gram-Schmidt orthogonalization method [41]. The computational results in [40] and [41] were on randomly generated data.…”

Tracking Many Solution Paths of a Polynomial Homotopy on a Graphics Processing Unit in Double Double and Quad Double Arithmetic

“…The linear systems in each Newton step we solve in the least squares sense via the modified Gram-Schmidt method. In [24] and [25] our computations were executed on randomly generated regular data sets. In [27], we integrated and improved the evaluation and differentiation codes to run Newton's method on some selected benchmark polynomial systems.…”

GPU Acceleration of Newton's Method for Large Systems of Polynomial Equations in Double Double and Quad Double Arithmetic

“…Because our computations are geared towards extended precision arithmetic which carry a higher cost per operation, we can afford a fine granularity in our parallel algorithms. Compared to our previous GPU implementations in [37,38], we have removed the restrictions on the dimensions and are now able to solve problems involving several thousands of variables. The performance investigation involves mixing the memory-bound polynomial evaluation and differentiation with the compute-bound linear system solving.…”