A novel MPI reduction algorithm resilient to imbalances in process arrival times

Marendić, Petar; Lemeire, Jan; Vucinic, Dean; Schelkens, Peter

doi:10.1007/s11227-016-1707-x

Cited by 8 publications

(25 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper concerns an article [9], provided by Marendic et al Its main subject is an MPI reduction algorithm whose main feature is the ability to mitigate an impact caused by imbalanced PAPs. The algorithm is dedicated for segmentable data and based on greedy scheduling of the segment transfers between the cooperating processes as soon as they are available for the reduce operation; the pseudocode is presented in Listing 1.…”

Section: Original Algorithmmentioning

confidence: 99%

“…Apart of the article: [9] being the subject of this paper, there is a number of other related works. The first comprehensive study over imbalanced PAPs was performed by Faraj et al in [6].…”

Section: Related Workmentioning

confidence: 99%

“…presented two new broadcast algorithms: one designed for block-Listing 1 Original Clairvoyant algorithm pseudocode. Source: [9] 1: Let M be a state matrix of size P • N . Set M( * , * ) = A, i.e.. mark all segments as available.…”

Section: Related Workmentioning

confidence: 99%

“…In [15,16] we proposed usage of an additional background thread to detect, predict and distribute PAT information over all cooperating processes while the computations are performed. As described above, Clairvoyant did not provide any out-of-the-box method of PAP estimation [9], which was our motivation to extend it with the background thread, see Sect. 6.…”

Section: Related Workmentioning

confidence: 99%

“…A novel PAP-aware algorithm was presented in [9] by Marendic et al It is dedicated to the MPI reduce operation, which makes the data vectors initially placed in the cooperating processes, to be reduced into one value by a defined math operation, e.g. sum, and placed in a root process.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Improving Clairvoyant: reduction algorithm resilient to imbalanced process arrival patterns

Proficz

Ocetkiewicz

2020

J Supercomput

View full text Add to dashboard Cite

The Clairvoyant algorithm proposed in “A novel MPI reduction algorithm resilient to imbalances in process arrival times” was analyzed, commented and improved. The comments concern handling certain edge cases in the original pseudocode and description, i.e., adding another state of a process, improved cache friendliness more precise complexity estimations and some other issues improving the robustness of the algorithm implementation. The proposed improvements include skipping of idle loop rounds, simplifying generation of the ready set and management of the state array and an about 90-fold reduction in memory usage. Finally an extension enabling process arrival times (PATs) prediction was added: an additional background thread used to exchange the data with the PAT estimations. The performed tests, with a dedicated mini-benchmark executed in an HPC environment, showed correctness and improved performance of the solution, with comparison to the original or other state-of-the-art algorithms.

show abstract

Section: Original Algorithmmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Improving Clairvoyant: reduction algorithm resilient to imbalanced process arrival patterns

Proficz

Ocetkiewicz

2020

J Supercomput

View full text Add to dashboard Cite

show abstract

Tree‐based fault‐tolerant collective operations for MPI

Margolin

Barak

2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary With the increase in size and complexity of high‐performance computing systems, the probability of failures, and the cost of recovery grow. Parallel applications running on these systems should be able to continue running in spite of node failures at arbitrary times. Collective operations are essential for many parallel MPI applications, and are often the first to detect such failures. This work presents tree‐based fault‐tolerant collective operations, which combine fault detection and recovery as an integral part each operation. We do this by extending existing tree‐based algorithms, to allow for a collective operation to succeed despite failing nodes before or during its run. This differs from other approaches, where recovery takes place after a failure of such operations have failed. The article includes a comparison between the performance of the proposed algorithm and other approaches, as well as a simulator‐based analysis of performance at scale.

show abstract

Investigation into MPI All-Reduce Performance in a Distributed Cluster with Consideration of Imbalanced Process Arrival Patterns

Proficz

Sumionka

Skomiał

et al. 2020

Advanced Information Networking and Applications

View full text Add to dashboard Cite

A novel MPI reduction algorithm resilient to imbalances in process arrival times

Cited by 8 publications

References 36 publications

Improving Clairvoyant: reduction algorithm resilient to imbalanced process arrival patterns

Improving Clairvoyant: reduction algorithm resilient to imbalanced process arrival patterns

Tree‐based fault‐tolerant collective operations for MPI

Investigation into MPI All-Reduce Performance in a Distributed Cluster with Consideration of Imbalanced Process Arrival Patterns

Contact Info

Product

Resources

About