Practical parallelization of scientific applications with OpenMP, OpenACC and MPI

Abstract

This work aims at distilling a systematic methodology to modernize existing sequential scientific codes with a little re-designing e↵ort, turning an old codebase into modern code, i.e., parallel and robust code. We propose a semi-automatic methodology to parallelize scientific applications designed with a purely sequential programming mindset, possibly using global variables, aliasing, random number generators, and stateful functions. We demonstrate that the same methodology works for the parallelization in the shared memory model (via OpenMP), message passing model (via MPI), and General Purpose Computing on GPU model (via OpenACC). The method is demonstrated parallelizing four real-world sequential codes in the domain of physics and material science. The methodology itself has been distilled in collaboration with MSc students of the Parallel Computing course at the University of Torino, that applied it for the first time to the project works that they presented for the final exam of the course. Every year the course hosts some special lectures from industry representatives, who present how they use parallel computing and o↵er codes to be parallelized.

A Parallel Hybrid Testing Technique for Tri-Programming Model-Based Software Systems

“…This paper focuses on detecting errors in systems that use tri-programming models to increase parallelism in high-performance computing software, including exascale systems. One possible combination is Message Passing Interface (MPI) + OpenMP + OpenACC, which has been implemented in several studies [5][6][7]. To the best of our knowledge, no testing tool or technique has been developed to discover errors that may arise because of this integration.…”

“…As a result of its widespread use, it significantly contributes to the acceleration of computing. It has been implemented in several research in combination with OpenMP [8][9][10] and OpenACC [5,11,12].…”

“…OpenMP is a shared memory application programming interface. Developers are using it more frequently due to the obvious widespread use of multi-core devices and multi-threaded processors [2,5,[8][9][10]. Using OpenMP, work can be distributed among CPU-core-based threads, and data can be shared or made private for each thread as needed by the program.…”

“…OpenACC and OpenMP are not equivalent, and each of them has its own advantages and disadvantages. The references [5,11,[15][16][17][18][19][20] provide additional details regarding the comparison of the two programming models. Several of these studies show that OpenACC is efficient at using accelerators.…”

“…Many studies [1,5,[8][9][10][11][12] have shown the possible benefits of integrating more than one programming model through memory exploitation and accelerator utilization. This may increase throughput and improve performance.…”