Design and implementation of a high-performance MPI for C# and the common language infrastructure

Abstract

As high-performance computing enters the mainstream, parallel programming mechanisms (including the Message Passing Interface, or MPI) must be supported in new environments such as C# and the Common Language Infrastructure (CLI). Making effective use of MPI with the CLI requires an interface that reflects the highlevel object-oriented nature of C# and that also supports its programming idioms. However, for performance reasons, this highlevel functionality must ultimately be mapped to low-level native MPI libraries. In addition to abstraction penalty concerns, avoiding unwanted overhead in this mapping process is significantly complicated by the safety and portability features of the CLI virtual machine, such as garbage collection and just-in-time compilation. In this paper, we describe our approach to using features of C# and the CLI-such as reflection, unsafe code regions, and run-time code generation-to realize an elegant, yet highly efficient, C# interface to MPI. Experimental results demonstrate that there is no appreciable overhead introduced by our approach when compared to the native MS-MPI library.

Waiting for Godot? the right language abstractions for parallel programming should be here soon

“…The Parallel Dwarf project [7], provides an implementation of these programs in two languages: C# and C++. The C# implementation has a sequential implementation, one based on shared memory via the TPL, and one based on message passing via MPI (message passing abstractions) [8]. Likewise, the C++ implementation has a sequential and shared memory implementation using OpenMP.…”

Writing Parallel Libraries with MPI - Common Practice, Issues, and Extensions

“…Programming Model Libraries offer a different (often limited) programming model such as the master/slave model (e.g., ADLB [15]) or fine-grained objects (e.g., AP [24]). System and Utility Libraries offer helper functionality to interface different architectural subsystems that are often outside the scope of MPI (e.g., LibTopoMap [10], HDF5 [3]) or language bindings (e.g., Boost.MPI, C# [5]). …”

“…This enables the reception of dynamically-sized messages. However, this also creates problems in the context of multiple threads [5] since one thread can query the message and another thread can receive it (the queue is a global shared object). A matched probe call that removes the message from the queue while peeking has been proposed to MPI-3 to solve this problem [7].…”

Efficient MPI Support for Advanced Hybrid Programming Models

“…Similarly, an implementation of language bindings such as MPI.NET [6] must assume the general case (scenario 7) so using the fine grained locking approach likely entails a high cost.…”