Efficient MPI Support for Advanced Hybrid Programming Models

Hoefler, Torsten; Bronevetsky, Greg; Barrett, Brian; Supinski, Bronis R. de; Lumsdaine, Andrew

doi:10.1007/978-3-642-15646-5_6

Cited by 8 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiple efforts have been made in the past to improve specific aspects of the interaction between multi-threaded programming models and MPI [31][32][33].…”

Section: Related Workmentioning

confidence: 99%

Callback-based Completion Notification using MPI Continuations

Schuchart,

Samfass,

Niethammer

et al. 2021

Preprint

View full text Add to dashboard Cite

Asynchronous programming models (APM) are gaining more and more traction, allowing applications to expose the available concurrency to a runtime system tasked with coordinating the execution. While MPI has long provided support for multi-threaded communication and non-blocking operations, it falls short of adequately supporting APMs as correctly and efficiently handling MPI communication in different models is still a challenge. We have previously proposed an extension to the MPI standard providing operation completion notifications using callbacks, so-called MPI Continuations. This interface is flexible enough to accommodate a wide range of different APMs.In this paper, we present an extension to the previously described interface that allows for finer control of the behavior of the MPI Continuations interface. We then present some of our first experiences in using the interface in the context of different applications, including the NAS parallel benchmarks, the PaRSEC task-based runtime system, and a load-balancing scheme within an adaptive mesh refinement solver called ExaHyPE. We show that the interface, implemented inside Open MPI, enables lowlatency, high-throughput completion notifications that outperform solutions implemented in the application space.

show abstract

“…Multiple efforts have been made in the past to improve specific aspects of the interaction between multi-threaded programming models and MPI [31][32][33].…”

Section: Related Workmentioning

confidence: 99%

Callback-based Completion Notification using MPI Continuations

Schuchart,

Samfass,

Niethammer

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Several improvements to MPI's ability of handling multi-threaded communication has been proposed over the years, ranging from thread-safe probes [18] over thread-specific endpoints [11] to partitioned communication [16] and (most recently) the use of continuations [29,33]. Work on improved implementation support for multi-threaded MPI in general [28] and RMA in particular [17] has also been described.…”

Section: Related Workmentioning

confidence: 99%

Quo Vadis MPI RMA? Towards a More Efficient Use of MPI One-Sided Communication

Schuchart¹,

Niethammer²,

Gracia³

et al. 2021

Preprint

View full text Add to dashboard Cite

The MPI standard has long included one-sided communication abstractions through the MPI Remote Memory Access (RMA) interface. Unfortunately, the MPI RMA chapter in the 4.0 version of the MPI standard still contains both well-known and lesser known short-comings for both implementations and users, which lead to potentially non-optimal usage patterns. In this paper, we identify a set of issues and propose ways for applications to better express anticipated usage of RMA routines, allowing the MPI implementation to better adapt to the application's needs. In order to increase the flexibility of the RMA interface, we add the capability to duplicate windows, allowing access to the same resources encapsulated by a window using different configurations. In the same vein, we introduce the concept of MPI memory handles, meant to provide life-time guarantees on memory attached to dynamic windows, removing the overhead currently present in using dynamically exposed memory. We will show that our extensions provide improved accumulate latencies, reduced overheads for multi-threaded flushes, and allow for zero overhead dynamic memory window usage.

show abstract

“…Mechanisms for fine-grain locking have been proposed [11] [7], which in some case should improve scalability with the number of requests, but our solution pushes the concept further.…”

Section: Related Workmentioning

confidence: 99%

Scalability of the NewMadeleine Communication Library for Large Numbers of MPI Point-to-Point Requests

Denis

2019

2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)

View full text Add to dashboard Cite

New kinds of applications with lots of threads or irregular communication patterns which rely a lot on point-topoint MPI communications have emerged. It stresses the MPI library with potentially a lot of simultaneous MPI requests for sending and receiving at the same time. To deal with large numbers of simultaneous requests, the bottleneck lies in two main mechanisms: the tag-matching (the algorithm that matches an incoming packet with a posted receive request), and the progression engine. In this paper, we propose algorithms and implementations that overcome these issues so as to scale up to thousands of requests if needed. In particular our algorithms are able to perform constant-time tag-matching even with any-source and any-tag support. We have implemented these mechanisms in our New-Madeleine communication library. Through micro-benchmarks and computation kernel benchmarks, we demonstrate that our MPI library exhibits better performance than state-of-the-art MPI implementations in cases with many simultaneous requests.

show abstract

Efficient MPI Support for Advanced Hybrid Programming Models

Cited by 8 publications

References 6 publications

Callback-based Completion Notification using MPI Continuations

Callback-based Completion Notification using MPI Continuations

Quo Vadis MPI RMA? Towards a More Efficient Use of MPI One-Sided Communication

Scalability of the NewMadeleine Communication Library for Large Numbers of MPI Point-to-Point Requests

Contact Info

Product

Resources

About