Lessons Learned on MPI+Threads Communication

Zambre, Rohit; Chandramowlishwaran, Aparna

doi:10.1109/sc41404.2022.00082

Cited by 2 publications

(1 citation statement)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Modern task‐based parallel programming models are concerned because they allow users to abstract computation as tasks with fine‐grained parallelism. Besides, the paradigm can employ better hybrid schemes of multithreading + multiprocessing like MPI+X; 12‐14 where modern computing architectures support multiple CPU sockets, a socket has multiple cores, and each core hosts a single thread. For our context, task is defined as a code region with its data, and

Tcomm

is deployed using hybrid MPI+X.…”

Section: Introductionmentioning

confidence: 99%

From reactive to proactive load balancing for task‐based parallel applications in distributed memory machines

Chung

Weidendorfer

Fürlinger

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryLoad balancing is often a challenge in task‐parallel applications. The balancing problems are divided into static and dynamic. “Static” means that we have some prior knowledge about load information and perform balancing before execution, while “dynamic” must rely on partial information of the execution status to balance the load at runtime. Conventionally, work stealing is a practical approach used in almost all shared memory systems. In distributed memory systems, the communication overhead can make stealing tasks too late. To improve, people have proposed a reactive approach to relax communication in balancing load. The approach leaves one dedicated thread per process to monitor the queue status and offload tasks reactively from a slow to a fast process. However, reactive decisions might be mistaken in high imbalance cases. First, this article proposes a performance model to analyze reactive balancing behaviors and understand the bound leading to incorrect decisions. Second, we introduce a proactive approach to improve further balancing tasks at runtime. The approach exploits task‐based programming models with a dedicated thread as well, namely . Nevertheless, the main idea is to force not only to monitor load; it will characterize tasks and train load prediction models by online learning. “Proactive” indicates offloading tasks before each execution phase proactively with an appropriate number of tasks at once to a potential victim (denoted by an underloaded/fast process). The experimental results confirm speedup improvements from to in important use cases compared to the previous solutions. Furthermore, this approach can support co‐scheduling tasks across multiple applications.

show abstract

Tcomm

is deployed using hybrid MPI+X.…”

Section: Introductionmentioning

confidence: 99%

From reactive to proactive load balancing for task‐based parallel applications in distributed memory machines

Chung

Weidendorfer

Fürlinger

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

Design and performance evaluation of UCX for the Tofu Interconnect D on Fugaku towards efficient multithreaded communication

Watanabe,

Tsuji,

Murai

et al. 2024

J Supercomput

View full text Add to dashboard Cite

The increasing trend of manycore processors makes multithreaded communication more important to avoid costly global synchronization among cores. One of the representative approaches that require multithreaded communication is the global task-based programming model. In the model, a program is divided into tasks, and tasks are asynchronously executed by each node, and independent thread-to-thread communications are expected. However, the Message passing interface (MPI) based approach is not efficient because of design issues. In this research, we design and implement the utofu transport layer in an abstracted communication library called Unified communication-X (UCX) for efficient remote direct memory access (RDMA) based multithreaded communication on Tofu Interconnect D. The evaluation results on Fugaku show that UCX can significantly improve the multithreaded performance over MPI, while maintaining portability between systems thanks to UCX. UCX shows about 32.8 times lower latency than Fujitsu MPI with 24 threads in the multithreaded pingpong benchmark and about 37.8 times higher update rate than Fujitsu MPI with 24 threads on 256 nodes in multithreaded GUPs benchmark.

show abstract

Lessons Learned on MPI+Threads Communication

Cited by 2 publications

References 49 publications

From reactive to proactive load balancing for task‐based parallel applications in distributed memory machines

From reactive to proactive load balancing for task‐based parallel applications in distributed memory machines

Design and performance evaluation of UCX for the Tofu Interconnect D on Fugaku towards efficient multithreaded communication

Contact Info

Product

Resources

About