The Effect of the 2 nd Generation Clusters: Changes in the Parallel Programming Paradigms

Abstract

Abstract. Programming paradigms for networks of symmetric multiprocessor (SMP) workstation (2 nd generation of clusters) are discussed and a new paradigm is introduced. The SMP cluster environments are explored in regard to their advantages and drawbacks with a special focus on memory architectures and communication. The new programming paradigm provides a solution to write efficient parallel applications for the 2 nd generation of clusters. The paradigm aims at improving the overlap of computation and communication and the locality of communication operations. The preliminary results with large message sizes indicate improvements in excess of 30% over traditional MPI implementations.

Reducing the Overhead of Intra-Node Communication in Clusters of SMPs

“…In [1] the authors proposed a thread-only implementation of MPI and it aims at the development stage of program where tests are performed on a single machine. The work in [10,14] goes one step further and rewrites parts of MPICH to shift the original process-only model to a thread-only model. The disadvantage of these approaches is the dependency on the operating system and the MPICH version.…”