Implementation of a non‐linear solver on heterogeneous architectures

Abstract

Heterogeneous architectures seem to be not only the present but also the future of the HPC world (eg, see the Exascale Project of U.S. Department of Energy or the European Horizon 2020 FET Proactive -High Performance Computing Call). A lot of work has been done in developing software libraries useful to solve problem described by linear equations on such computing systems.Instead, not the same effort is spent in such context for the implementation of software modules to be used to solve non-linear problem. In this work, we present some experiences related with the implementation of a Quasi-Newton method able to exploit, using a combination of "Task Scheduling," "matrix-free," and "look-ahead" approaches, both the CPUs and the GP-GPUs components of a heterogeneous system.

Toward a new linpack‐like benchmark for heterogeneous computing resources

“…These tools should be capable of evaluating how well the computing environment can deliver the high performance required by SC applications. [17][18][19][20] The tests performed by these tools are often used for both assessing the overall system performance and comparing the performance of different systems. For instance, the Linpack benchmark is a high-level benchmark used to evaluate the performance of computing systems in terms of their ability to process large-scale problems.…”

“…Red, green, and blue lines respectively represent the Implementation of the Schur-Based algorithm, slate::gesv on GPU, and slate::gesv on CPU test results. It follows, from (15) and (19), that to get computational load balance, a good choice for 𝛼 CPU (and then of n 1 ) should be such that O(Γ(𝛼 CPU )) = 10 −3 . From Figure 3, we can observe that it happens when 0.06 < 𝛼 CPU < 0.18 (see the zoomed part of the plot).…”

“…In the field of Scientific Computing (SC), high‐level benchmarks are designed to test the overall system performance, including the utilization of the CPU, memory, and hard drive, in conjunction with all available computing devices (i.e., GPUs). These tools should be capable of evaluating how well the computing environment can deliver the high performance required by SC applications 17‐20 …”

Models, algorithms, and tools for highly heterogeneous computing environments

“…The paper “Implementation of a non‐linear solver on heterogeneous architectures” by Carracciuolo and Lapegna presents some experiences related with the implementation of a Quasi‐Newton method able to exploit, using a combination of “task‐scheduling,” “matrix‐free,” and “look‐ahead” approaches, both the CPUs and the GP‐GPUs components of a heterogeneous system. The strategies adopted for the implementations of the algorithm are based on the representation of the algorithm as a collection of BLAS‐based tasks and dependencies among them, on a proper scheduling of tasks in the heterogeneous environment, and on the implementation of some mechanism to overlap CPU and GPU work.…”

About the granularity portability of block‐based Krylov methods in heterogeneous computing environments

“…Most likely, these systems will respond to the following description: multinode systems where each node will have a high level of internal parallelism which will be also made available by technologies such as NVIDIA GPU and Intel Xeon Phi. In particular, as in Carracciuolo and Lapegna 21 this work intends to analyze the portability of some performance metrics of a parallel implementation of a KM block‐based algorithm on heterogeneous CPU–GPU systems equipped with standard scientific libraries as MAGMA 12 . The considered implementation proposes novel revisitation of well‐known algorithms, such as those used to compute the QR factorization, which is based on the overlapping of communication and computation.…”