Pipelining and overlapping for mpi collective operations

Abstract

Abstract

Implementation and analysis of nonblocking collective operations on SCI networks

“…[9] describes the combination of DMA and PIO in pipelining algorithms for Broadcast, Reduce, and Allreduce. We used the ideas presented in [10] to tailor our algorithms for torus networks.…”

ATCOM: Automatically Tuned Collective Communication System for SMP Clusters

“…Current collective communication algorithms [19,20,21,22,23,24,15,16,17,25,26,27] are designed for the inter-node layer, and are point-to-point, based only. Some algorithms consider the features of SMP architecture [18] ; the approach is also based on point-to-point communications, and no overlapping between inter-node/intra-node communications.…”

“…Different algorithms for collective communications on tlie inter-node layer have been developed during the last two decades [20,21,22,24,23,15,16,17,25,26,27]. Their assumption is that there is no hierarchical structure in communications, and the underlying network is fully connected.…”

Two Algorithms of Irregular Scatter/Gather Operations for Heterogeneous Platforms

“…Optimization of collectives is not a new research topic and a wide range of optimized collective algorithms have been proposed in the past [1,3,5]. Communication performance models [4] have been used for collectives by predicting the runtime of various collective algorithms and switching between them accordingly [6].…”