Two Algorithms of Irregular Scatter/Gather Operations for Heterogeneous Platforms

Dichev, Kiril; Рычков, В. Н.; Lastovetsky, Alexey

doi:10.1007/978-3-642-15646-5_31

Cited by 13 publications

(4 citation statements)

References 7 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second set of experiments was performed on relatively homogeneous nodes. We took 90 nodes of similar per-core performance from eight clusters: Genepi (7), Edel (11), Griffon (16), Graphene (19), Chinqchint (17), Chimint (13), Orion (4), and Taurus (3). We spawned one single-threaded MPI process per node to even the processing speeds as much as possible.…”

Section: Experiments On Relatively Homogeneous Nodesmentioning

confidence: 99%

See 1 more Smart Citation

Network‐aware optimization of communications for parallel matrix multiplication on hierarchical HPC platforms

Malik

Рычков

Lastovetsky

2015

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Summary Communications on hierarchical heterogeneous high‐performance computing platforms can be optimized based on topology and performance information. For MPI, as a major programming tool for such platforms, a number of topology‐aware and performance‐aware implementations of collective operations have been proposed for optimal scheduling of messages. This approach improves performance of application and does not require to modify application source code. However, it is applicable to collective operations only and does not affect the parts of the application that are based on point‐to‐point exchanges. In this paper, we address the problem of efficient execution of data‐parallel applications on interconnected clusters and present optimizations that improve data partition by taking into account the entire communication flow of the application. This approach is also non‐intrusive to the source code but application specific. For illustration, we use parallel matrix multiplication, where the matrices are partitioned into irregular two‐dimensional rectangles assigned to different processors and arranged in columns, and the processors communicate over this partition vertically and horizontally. By rearranging the rectangles, we can minimize communications between different levels of the network hierarchy. Finding the optimal arrangement is NP‐complete; therefore, we propose two heuristic approaches based on evaluation of the communication flow on the given network topology. We demonstrate the correctness and efficiency of the proposed approaches by experimental results on multicore nodes and interconnected heterogeneous clusters. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

Section: Experiments On Relatively Homogeneous Nodesmentioning

confidence: 99%

“…One approach is to choose the collective algorithm with the minimal predicted time from a given set of algorithms . Another approach is to use the predictions for building more optimal communication trees for a collective communication .…”

Section: Related Workmentioning

confidence: 99%

Network‐aware optimization of communications for parallel matrix multiplication on hierarchical HPC platforms

Malik

Рычков

Lastovetsky

2015

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Other heuristics of trees with binomial or other structure also exist, for example considering overlap of communication [25]. Interestingly, it is not always the case that complex heuristics result in better efficiency -some evidence suggests that even a simple heuristic based on a fixed tree structure with reordering of processes can produce efficient communication trees [26].…”

Section: Homogeneous Performance Modelsmentioning

confidence: 99%

Optimization of Collective Communication for Heterogeneous HPC Platforms

Dichev

Lastovetsky

2014

High‐Performance Computing on Complex Environments

Self Cite

View full text Add to dashboard Cite

“…Heuristics can alleviate this problem, and early work computing communication trees in polynomial time includes [9,2,1]. More recently, we have used models to also generate efficient communication trees [7]. As an alternative, simpler broadcast algorithms can be devised to reflect e.g.…”

Section: Introduction and Related Workmentioning

confidence: 99%

MPI vs. BitTorrent: Switching between Large-Message Broadcast Algorithms in the Presence of Bottleneck Links

Dichev

Lastovetsky

2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Collective communication in high-performance computing is traditionally implemented as a sequence of point-to-point communication operations. For example, in MPI a broadcast is often implemented using a linear or binomial tree algorithm. These algorithms are inherently unaware of any underlying network heterogeneity. Integrating topology awareness into the algorithms is the traditional way to address this heterogeneity, and it has been demonstrated to greatly optimize tree-based collectives. However, recent research in distributed computing shows that in highly heterogeneous networks an alternative class of collective algorithms -BitTorrent-based multicasts -has the potential to outperform topology-aware tree-based collective algorithms. In this work, we experimentally compare the performance of BitTorrent and tree-based largemessage broadcast algorithms in a typical heterogeneous computational cluster. We address the following question: Can the dynamic data exchange in BitTorrent be faster than the static data distribution via trees even in the context of high-performance computing? We find that both classes of algorithms have a justification of use for different settings. While on single switch clusters linear tree algorithms are optimal, once multiple switches and a bottleneck link are introduced, BitTorrent broadcasts -which utilize the network in a more adaptive way -outperform the tree-based MPI implementations.

show abstract

Two Algorithms of Irregular Scatter/Gather Operations for Heterogeneous Platforms

Cited by 13 publications

References 7 publications

Network‐aware optimization of communications for parallel matrix multiplication on hierarchical HPC platforms

Network‐aware optimization of communications for parallel matrix multiplication on hierarchical HPC platforms

Optimization of Collective Communication for Heterogeneous HPC Platforms

MPI vs. BitTorrent: Switching between Large-Message Broadcast Algorithms in the Presence of Bottleneck Links

Contact Info

Product

Resources

About