Optimizing MPI Runtime Parameter Settings by Using Machine Learning

Pellegrini, Simone; Wang, Jie; Fahringer, Thomas; Moritsch, H

doi:10.1007/978-3-642-03770-2_26

Cited by 20 publications

(11 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The OPTO framework [18] tests all possible configuration combinations so as to automatically find the best one. Machine learning was also proposed as an alternative method [19]. A training tool finds out important characteristics of the platform before matching them with specific application needs.…”

Section: E Shared-buffer Reuse Order and Moesi Protocolmentioning

confidence: 99%

Analysis of MPI Shared-Memory Communication Performance from a Cache Coherence Perspective

Putigny

Ruelle

Goglin

2014

2014 IEEE International Parallel &Amp; Distributed Processing Symposium Workshops

View full text Add to dashboard Cite

Shared memory MPI communication is an important part of the overall performance of parallel applications. However understanding the behavior of these data transfers is difficult because of the combined complexity of modern memory architectures with multiple levels of caches and complex cache coherence protocols, of MPI implementations, and of application needs.We analyze shared memory MPI communication from a cache coherence perspective through a new memory model. It captures the memory architecture characteristics with microbenchmarks that exhibit the limitations of the memory accesses involved in the data transfer. We model the performance of intra-node communication without requiring complex analytical models. The advantage of the approach consists in not requiring deep knowledge of rarely documented hardware features such as caching policies or prefetchers that make modeling modern memory subsystems hardly feasible.Our qualitative analysis based on this result leads to a better understanding of shared memory communication performance for scientific computing. We then discuss some possible optimizations such as buffer reuse order, cache flushing, and nontemporal instructions that could be used by MPI implementers.

show abstract

Section: E Shared-buffer Reuse Order and Moesi Protocolmentioning

confidence: 99%

Analysis of MPI Shared-Memory Communication Performance from a Cache Coherence Perspective

Putigny

Ruelle

Goglin

2014

2014 IEEE International Parallel &Amp; Distributed Processing Symposium Workshops

View full text Add to dashboard Cite

show abstract

“…Mohamad C. et al [6] proposed OTPO, a tool that can optimize OpenMPI runtime parameters, which gives users and system researchers the possibility to make their environment meet the requirement of performance. In [28,29], the main idea is to conduct an off-line training phase to derive the "best" configurations of OpenMPI for each target architecture based on machine learning algorithms. Jha et.al.…”

Section: Related Workmentioning

confidence: 99%

Auto-Tuning MPI Collective Operations on Large-Scale Parallel Systems

Zheng

Fang

Chen

et al. 2019

2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference On

View full text Add to dashboard Cite

MPI libraries are widely used in applications of high performance computing. Yet, effective tuning of MPI colletives on large parallel systems is an outstanding challenge. This process often follows a trial-and-error approach and requires expert insights into the subtle interactions between software and the underlying hardware. This paper presents an empirical approach to choose and switch MPI communication algorithms at runtime to optimize the application performance. We achieve this by first modeling offline, through microbenchmarks, to find how the runtime parameters with different message sizes affect the choice of MPI communication algorithms. We then apply the knowledge to automatically optimize new unseen MPI programs. We evaluate our approach by applying it to NPB and HPCC benchmarks on a 384-node computer cluster of the Tianhe-2 supercomputer. Experimental results show that our approach achieves, on average, 22.7% (up to 40.7%) improvement over the default setting.

show abstract

“…Much work has been done on creating machine learning‐based models and using them for tuning and auto‐tuning, eg, to determine loop unroll factors, which optimizations to apply for parallel stencil computations, Message Passing Interface (MPI) parameters, and general compiler optimizations . Kulkarni et al developed a method to determine a good ordering of compiler optimization phases, on a per function basis.…”

Section: Related Workmentioning

confidence: 99%

“…Furthermore, analytical performance models for GPUs and heterogeneous systems have been developed [42][43][44][45] and used for auto-tuning. 5 Much work has been done on creating machine learning-based models and using them for tuning and auto-tuning, eg, to determine loop unroll factors, 28,46 which optimizations to apply for parallel stencil computations, 7,47 Message Passing Interface (MPI) parameters, 14 and general compiler optimizations. [20][21][22][23][24][25][26][27][28][29][30][31][32] Kulkarni et al 13 Machine learning approaches have also been used for performance modeling and auto-tuning in a heterogeneous setting.…”

Section: Related Workmentioning

confidence: 99%

Machine learning‐based auto‐tuning for enhanced performance portability of OpenCL applications

Falch

Elster

2016

Concurrency and Computation

View full text Add to dashboard Cite

Summary Heterogeneous computing, combining devices with different architectures such as CPUs and GPUs, is rising in popularity and promises increased performance combined with reduced energy consumption. OpenCL has been proposed as a standard for programming such systems and offers functional portability. However, it suffers from poor performance portability, because applications must be retuned for every new device. In this paper, we use machine learning‐based auto‐tuning to address this problem. Benchmarks are run on a random subset of the tuning parameter spaces, and the results are used to build a machine learning‐based performance model. The model can then be used to find interesting subspaces for further search. We evaluate our method using five image processing benchmarks, with tuning parameter space sizes up to 2.3 M, using different input sizes, on several devices, including an Intel i7 4771 (Haswell) CPU, an Nvidia Tesla K40 GPU, and an AMD Radeon HD 7970 GPU. We compare different machine learning algorithms for the performance model. Our model achieves a mean relative error as low as 3.8% and is able to find solutions on average only 0.29% slower than the best configuration in some cases, evaluating less than 1.1% of the search space. The source code of our framework is available at https://github.com/acelster/ML‐autotuning.

show abstract

Optimizing MPI Runtime Parameter Settings by Using Machine Learning

Cited by 20 publications

References 2 publications

Analysis of MPI Shared-Memory Communication Performance from a Cache Coherence Perspective

Analysis of MPI Shared-Memory Communication Performance from a Cache Coherence Perspective

Auto-Tuning MPI Collective Operations on Large-Scale Parallel Systems

Machine learning‐based auto‐tuning for enhanced performance portability of OpenCL applications

Contact Info

Product

Resources

About