Hardware MPI message matching: Insights into MPI matching behavior to inform design

Ferreira, Kurt Brian; Grant, Ryan E.; Levenhagen, Michael J.; Levy, Scott; Groves, Taylor

doi:10.1002/cpe.5150

Cited by 6 publications

(3 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the hash table data structure, we decided to use the uthash library 1 . It is an open-source implementation of a hash table that uses the separate chaining technique [15] for handling collisions.…”

Section: Algorithm 1 Hash Table Matchingmentioning

confidence: 99%

See 1 more Smart Citation

Performance Improvements of Parallel Applications thanks to MPI-4.0 Hints

Moraru

Roussel

Taboada

et al. 2022

2022 IEEE 34th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)

View full text Add to dashboard Cite

HPC systems have experienced significant growth over the past years, with modern machines having hundreds of thousands of nodes. Message Passing Interface (MPI) is the de facto standard for distributed computing on these architectures. On the MPI critical path, the message-matching process is one of the most time-consuming operations. In this process, searching for a specific request in a message queue represents a significant part of the communication latency. So far, no miracle algorithm performs well in all cases. This paper explores potential matching specializations thanks to hints introduced in the latest MPI 4.0 standard. We propose a hash-table-based algorithm that performs constant time message-matching for no wildcard requests. This approach is suitable for intensive point-to-point communication phases in many applications (more than 50% of CORAL benchmarks). We demonstrate that our approach can improve the overall execution time of real HPC applications by up to 25%. Also, we analyze the limitations of our method and propose a strategy for identifying the most suitable algorithm for a given application. Indeed, we apply machine learning techniques for classifying applications depending on their message pattern characteristics.

show abstract

Section: Algorithm 1 Hash Table Matchingmentioning

confidence: 99%

“…This operation is usually called MPI matching and is performed on the receiver side. MPI matching is on the critical path and is known to be computationally complex [1], [2]. Thus, its performance significantly impacts the message latency [3].…”

Section: Introductionmentioning

confidence: 99%

Performance Improvements of Parallel Applications thanks to MPI-4.0 Hints

Moraru

Roussel

Taboada

et al. 2022

2022 IEEE 34th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)

View full text Add to dashboard Cite

show abstract

“…Flowchart highlighting key transitions in MPI message matching for source and target process. Original figure from work by Ferreira et al 7 …”

Section: Introductionmentioning

confidence: 99%

Not all applications have boring communication patterns: Profiling message matching with BMM

Groves

Ravichandrasekaran²,

Cook

et al. 2021

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Message matching within MPI is an important performance consideration for applications that utilize two-sided semantics. In this work, we present an instrumentation of the CrayMPI library that allows the collection of detailed message-matching statistics as well as an implementation of hashed matching in software. We use this functionality to profile key DOE applications with complex communication patterns to determine under what circumstances an application might benefit from hardware offload capabilities within the NIC to accelerate message matching. We find that there are several applications and libraries that exhibit sufficiently long match list lengths to motivate a Binned Message Matching approach.

show abstract