A Parallel Task-Based Approach to Linear Algebra

Abstract

Abstract-Processors with large numbers of cores are becoming commonplace. In order to take advantage of the available resources in these systems, the programming paradigm has to move towards increased parallelism. However, increasing the level of concurrency in the program does not necessarily lead to better performance. Parallel programming models have to provide flexible ways of defining parallel tasks and at the same time, efficiently managing the created tasks. OpenMP is a widely accepted programming model for shared-memory architectures. In this paper we highlight some of the drawbacks in the OpenMP tasking approach, and propose an alternative model based on the Glasgow Parallel Reduction Machine (GPRM) programming framework. As the main focus of this study, we deploy our model to solve a fundamental linear algebra problem, LU factorisation of sparse matrices. We have used the SparseLU benchmark from the BOTS benchmark suite, and compared the results obtained from our model to those of the OpenMP tasking approach. The TILEPro64 system has been used to run the experiments. The results are very promising, not only because of the performance improvement for this particular problem, but also because they verify the task management efficiency, stability, and flexibility of our model, which can be applied to solve problems in future many-core systems.

Locality-Aware Task Scheduling and Data Distribution for OpenMP Programs on NUMA Systems and Manycore Processors

“…Tousimojarad and Vanderbauwhede [33] cleverly reduce access latencies to uniformly distributed data by using copies whose home cache is local to the access thread on the TILEPro64 processor. Zhou and Demsky [2] build a NUMAaware adaptive garbage collector that migrate objects to improve locality on manycore processors.…”

Comparison of Three Popular Parallel Programming Models on the Intel Xeon Phi

“…Deriving the parallel kernel from the generated single-threaded code is mostly a matter of replacing the loops by the OpenCL indexing calls (get global id, get local id etc), and in the case where the original code has multiple loops, as is the case 2 [omitted for blind review] 3…”

“…Since all the tasks defined in the GPC code will be executed in parallel, a seq pragma is required to run the two phases sequentially. Each phase uses a partial continuous for, par cont for [3], in order to parallelise the outer loop over rows, and a #pragma simd 4 to help the compiler vectorise the inner loop over columns. par cont for is a sequential for loop that works as follows:…”

“…We then manually optimise the kernel and if required the OpenCL API calls. The OpenCL API we use is our own OpenCL wrapper library 3 , which provides convenient OpenCL integration in existing codebases for C, C++ and Fortran.…”

“…For this purpose, we have chosen three parallel programming models from different domains and for different reasons: OpenMP, the de-facto standard for programming shared memory architectures; OpenCL, known for being portable across multiple platforms; and finally GPRM, a pure taskbased programming framework. It has been reported that GPRM provides superior performance compared to OpenMP on manycore architectures [3] [4].…”