Model-Based Estimation of the Communication Cost of Hybrid Data-Parallel Applications on Heterogeneous Clusters

Rico‐Gallego, Juan‐Antonio; Lastovetsky, Alexey; Díaz-Martín, Juan-Carlos

doi:10.1109/tpds.2017.2715809

Cited by 21 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The accurate measurement of energy consumption during workload execution plays a crucial role in energy-aware HPC computing [30], [31]. There are two popular approaches in this area [12], [32]: (i) hardware-based technique that uses an external AC power meter, and (ii) software-based approach that explores on-chip power sensors. The first approach enables monitoring of power and energy for the whole computing system, using an external AC power meter [30] such as Yokogawa WT310 digital power meter.…”

Section: Measurements Techniques For Power and Energymentioning

confidence: 99%

Correlation of Performance Optimizations and Energy Consumption for Stencil-Based Application on Intel Xeon Scalable Processors

Szustak

Wyrzykowski

Olas

et al. 2020

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

This article provides a comprehensive study of the impact of performance optimizations on the energy efficiency of a real-world CFD application called MPDATA, as well as an insightful analysis of performance-energy interaction of these optimizations with the underlying hardware that represents the first generation of Intel Xeon Scalable processors. Considering the MPDATA iterative application as a use case, we explore the fundamentals of energy and performance analysis for a memory-bound application when exposed to a set of optimization steps that increase the application performance, by improving the operational intensity of code and utilizing resources more efficiently. It is shown that for memory-bound applications, optimizing toward high performance could be a powerful strategy for improving the energy efficiency as well. In fact, for the considered performance optimizations, the energy gain is correlated with the performance gain but with varying degrees. As a result, these optimizations allow improving both performance and energy consumption radically, up to about 10.9 and 8.8 times, respectively. The impact of the Intel AVX-512 SIMD extension on the energy consumption and performance is demonstrated. Also, we discover limitations on the usability of CPU frequency scaling as a tool for balancing energy savings with admissible performance losses.

show abstract

Section: Measurements Techniques For Power and Energymentioning

confidence: 99%

Correlation of Performance Optimizations and Energy Consumption for Stencil-Based Application on Intel Xeon Scalable Processors

Szustak

Wyrzykowski

Olas

et al. 2020

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…In our work we formulate some restrictions that are connected to energy efficiency as wellwe do not use additional available computational resources if the parallelisation efficiency drops below some specific level. The other work [30] is dedicated to model-based optimisation on hybrid heterogeneous systems composed of CPUs and accelerators. In that research authors investigate the problem of communications costs due to uneven workload distribution between accelerators and CPUs.…”

Section: The Related Workmentioning

confidence: 99%

A Three-Level Parallelisation Scheme and Application to the Nelder-Mead Algorithm

Kriauzienė

Bugajev

Čiegis

2020

Mathematical Modelling and Analysis

View full text Add to dashboard Cite

We consider a three-level parallelisation scheme. The second and third levels define a classical two-level parallelisation scheme and some load balancing algorithm is used to distribute tasks among processes. It is well-known that for many applications the efficiency of parallel algorithms of these two levels starts to drop down after some critical parallelisation degree is reached. This weakness of the twolevel template is addressed by introduction of one additional parallelisation level. s an alternative to the basic solver some new or modified algorithms are considered on this level. The idea of the proposed methodology is to increase the parallelisation degree by using possibly less efficient algorithms in comparison with the basic solver. As an example we investigate two modified Nelder-Mead methods. For the selected application, a Schro¨dinger equation is solved numerically on the second level, and on the third level the parallel Wang’s algorithm is used to solve systems of linear equations with tridiagonal matrices. A greedy workload balancing heuristic is proposed, which is oriented to the case of a large number of available processors. The complexity estimates of the computational tasks are model-based, i.e. they use empirical computational data.

show abstract

“…The cost of data movement has turned into the dominant factor in HPC systems (Malik et al, 2016; Rico-Gallego et al, 2017). To reduce this cost, applications have to be redesigned and tuned for data movements both in the memory hierarchy and between processing units (Hager and Wellein, 2011; Wyrzykowski et al, 2012a).…”

Section: Related Workmentioning

confidence: 99%

“…Multi- and manycore architectures of emerging microprocessor designs (Ang et al, 2014; Rico-Gallego et al, 2017) have become increasingly complex, hierarchical, and heterogeneous (Malik et al, 2016; Rojek and Szustak, 2012; Szustak et al, 2017). With the quick development of computing platforms and software environments, application developers are forced to contend with a variety of parallel architectures.…”

Section: Introductionmentioning

confidence: 99%

Performance portable parallel programming of heterogeneous stencils across shared-memory platforms with modern Intel processors

Szustak

Bratek

2019

The International Journal of High Performance Computing Applica

View full text Add to dashboard Cite

In this work, we take up the challenge of performance portable programming of heterogeneous stencil computations across a wide range of modern shared-memory systems. An important example of such computations is the Multidimensional Positive Definite Advection Transport Algorithm (MPDATA), the second major part of the dynamic core of the EULAG geophysical model. For this aim, we develop a set of parametric optimization techniques and four-step procedure for customization of the MPDATA code. Among these techniques are: islands-of-cores strategy, (3+1)D decomposition, exploiting data parallelism and simultaneous multithreading, data flow synchronization, and vectorization. The proposed adaptation methodology helps us to develop the automatic transformation of the MPDATA code to achieve high sustained scalable performance for all tested ccNUMA platforms with Intel processors of last generations. This means that for a given platform, the sustained performance of the new code is kept at a similar level, independently of the problem size. The highest performance utilization rate of about 41–46% of the theoretical peak, measured for all benchmarks, is provided for any of the two-socket servers based on Skylake-SP (SKL-SP), Broadwell, and Haswell CPU architectures. At the same time, the four-socket server with SKL-SP processors achieves the highest sustained performance of around 1.0–1.1 Tflop/s that corresponds to about 33% of the peak.

show abstract

Model-Based Estimation of the Communication Cost of Hybrid Data-Parallel Applications on Heterogeneous Clusters

Cited by 21 publications

References 31 publications

Correlation of Performance Optimizations and Energy Consumption for Stencil-Based Application on Intel Xeon Scalable Processors

Correlation of Performance Optimizations and Energy Consumption for Stencil-Based Application on Intel Xeon Scalable Processors

A Three-Level Parallelisation Scheme and Application to the Nelder-Mead Algorithm

Performance portable parallel programming of heterogeneous stencils across shared-memory platforms with modern Intel processors

Contact Info

Product

Resources

About