A Multithreaded Message Passing Interface (MPI) Architecture: Performance and Program Issues

Protopopov, Boris V.; Skjellum, Anthony

doi:10.1006/jpdc.2000.1674

Cited by 38 publications

(15 citation statements)

References 3 publications

(5 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All message passing communication is performed outside of the parallel region (lines 5-9 and 20-23), while the multi-threading parallel computation occurs in lines [10][11][12][13][14][15][16][17][18][19]. Note that no explicit barrier is required for thread synchronization, as this effect is implicitly achieved by exiting the multi-threading parallel region.…”

Section: Fine-grain Hybrid Modelmentioning

confidence: 99%

“…Solid multi-threading support from the message passing library is also a very important issue; often, achieving high performance under hybrid programming is directly associated with overcoming the limitations imposed by the message passing library. Towards this direction, thread-safe extensions to popular existing message passing libraries, that would allow for portability, overlapping of computation and communication, and preserving latencies of individual networks, such as the approach presented in [17], could be helpful.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

The Effect of Process Topology and Load Balancing on Parallel Programming Models for SMP Clusters and Iterative Algorithms

Drosinos

Koziris

2006

J Supercomput

View full text Add to dashboard Cite

This article focuses on the effect of both process topology and load balancing on various programming models for SMP clusters and iterative algorithms. More specifically, we consider nested loop algorithms with constant flow dependencies, that can be parallelized on SMP clusters with the aid of the tiling transformation. We investigate three parallel programming models, namely a popular message passing monolithic parallel implementation, as well as two hybrid ones, that employ both message passing and multi-threading. We conclude that the selection of an appropriate mapping topology for the mesh of processes has a significant effect on the overall performance, and provide an algorithm for the specification of such an efficient topology according to the iteration space and data dependencies of the algorithm. We also propose static load balancing techniques for the computation distribution between threads, that diminish the disadvantage of the master thread assuming all inter-process communication due to limitations often imposed by the message passing library. Both improvements are implemented as compile-time optimizations and are further experimentally evaluated. An overall comparison of the above parallel programming styles on SMP clusters based on micro-kernel experimental evaluation is further provided, as well.

show abstract

Section: Fine-grain Hybrid Modelmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

The Effect of Process Topology and Load Balancing on Parallel Programming Models for SMP Clusters and Iterative Algorithms

Drosinos

Koziris

2006

J Supercomput

View full text Add to dashboard Cite

show abstract

“…Multithreaded and thread-safe MPI implementations have been proposed in [1], [2], [11], [12], [14], [15] 2. Hybrid models with MPI and OpenMP have been studied in [4], [6], [7], [8], [13] The programming paradigm, MPIT, developed by the authors is a communication library built on top of MPI.…”

Section: New 2-level Communication Paradigmmentioning

confidence: 99%

The Effect of the 2 nd Generation Clusters: Changes in the Parallel Programming Paradigms

Porras

Huttunen

Ikonen

2004

Computational Science - ICCS 2004

View full text Add to dashboard Cite

Abstract. Programming paradigms for networks of symmetric multiprocessor (SMP) workstation (2 nd generation of clusters) are discussed and a new paradigm is introduced. The SMP cluster environments are explored in regard to their advantages and drawbacks with a special focus on memory architectures and communication. The new programming paradigm provides a solution to write efficient parallel applications for the 2 nd generation of clusters. The paradigm aims at improving the overlap of computation and communication and the locality of communication operations. The preliminary results with large message sizes indicate improvements in excess of 30% over traditional MPI implementations.

show abstract

“…The MPI-SIM project [Prakash and Bagrodia 1998] has used multi-threading to simulate MPI execution on distributed memory machines for performance prediction as we will discuss in Section 3.1. Thread safety of MPI systems is addressed in [MPI-Forum 1999;Protopopov and Skjellum 1998;Skjellum et al 1996] and recent commercial MPI products from SUN, IBM and SGI are thread-safe. However, their concern is how multiple threads can be invoked in each MPI node, but not how to execute each MPI node as a thread.…”

Section: Assumptions and Related Workmentioning

confidence: 99%

Program transformation and runtime support for threaded MPI execution on shared-memory machines

Tang

Shen

Yang

2000

ACM Trans. Program. Lang. Syst.

View full text Add to dashboard Cite

Parallel programs written in MPI have been widely used for developing high-performance applications on various platforms. Because of a restriction of the MPI computation model, conventional MPI implementations on shared memory machines map each MPI node to an OS process, which can suffer serious performance degradation in the presence of multiprogramming. This paper studies compile-time and runtime techniques for enhancing performance portability of MPI code running on multiprogrammed shared memory machines. The proposed techniques allow MPI nodes to be executed safely and efficiently as threads. Compile-time transformation eliminates global and static variables in C code using node-specific data. The runtime support includes an efficient and provablycorrect communication protocol that uses lock-free data structure and takes advantage of address space sharing among threads. The experiments on SGI Origin 2000 show that our MPI prototype called TMPI using the proposed techniques is competitive with SGI's native MPI implementation in a dedicated environment, and that it has significant performance advantages in a multiprogrammed environment.

show abstract

A Multithreaded Message Passing Interface (MPI) Architecture: Performance and Program Issues

Cited by 38 publications

References 3 publications

The Effect of Process Topology and Load Balancing on Parallel Programming Models for SMP Clusters and Iterative Algorithms

The Effect of Process Topology and Load Balancing on Parallel Programming Models for SMP Clusters and Iterative Algorithms

The Effect of the 2 nd Generation Clusters: Changes in the Parallel Programming Paradigms

Program transformation and runtime support for threaded MPI execution on shared-memory machines

Contact Info

Product

Resources

About