Scheduling Optimization of Parallel Linear Algebra Algorithms Using Supervised Learning

Laberge, Gabriel; Shirzad, Shahrzad; Diehl, Patrick; Kaiser, Hartmut; Prudhomme, Serge; Lemoine, Adrian S.

doi:10.1109/mlhpc49564.2019.00009

Cited by 5 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For details about the effect on performance of chunk sizes, we refer to [4]. A machine learning approach is presented here [8,7]. With respect to vectorization, HPX provides the execution policy hpx::execution::simd to execute the algorithm using vectorization.…”

Section: Parallel Algorithmsmentioning

confidence: 99%

Shared memory parallelism in Modern C++ and HPX

Diehl¹,

Brandt²,

Kaiser³

2023

Preprint

View full text Add to dashboard Cite

Parallel programming remains a daunting challenge, from the struggle to express a parallel algorithm without cluttering the underlying synchronous logic, to describing which devices to employ in a calculation, to correctness. Over the years, numerous solutions have arisen, many of them requiring new programming languages, extensions to programming languages, or the addition of pragmas. Support for these various tools and extensions is available to a varying degree. In recent years, the C++ standards committee has worked to refine the language features and libraries needed to support parallel programming on a single computational node. Eventually, all major vendors and compilers will provide robust and performant implementations of these standards. Until then, the HPX library and runtime provides cutting edge implementations of the standards, as well as proposed standards and extensions. Because of these advances, it is now possible to write high performance parallel code without custom extensions to C++. We provide an overview of modern parallel programming in C++, describing the language and library features, and providing brief examples of how to use them.

show abstract

Section: Parallel Algorithmsmentioning

confidence: 99%

Shared memory parallelism in Modern C++ and HPX

Diehl¹,

Brandt²,

Kaiser³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…On the other hand, Khatami et al in [57] recently used a logistic regression model for predicting the optimal chunk size for a scheduling strategy, combining CSS and work-stealing. Similarly, Laberge et al [58] propose a machine-learning based strategy for accelerating linear algebra applications. These supervisedlearning based approaches are limited in the sense that they…”

Section: Related Workmentioning

confidence: 99%

A Probabilistic Machine Learning Approach to Scheduling Parallel Loops with Bayesian Optimization

Kim,

Park

2022

Preprint

View full text Add to dashboard Cite

This paper proposes Bayesian optimization augmented factoring self-scheduling (BO FSS), a new parallel loop scheduling strategy. BO FSS is an automatic tuning variant of the factoring self-scheduling (FSS) algorithm and is based on Bayesian optimization (BO), a black-box optimization algorithm. Its core idea is to automatically tune the internal parameter of FSS by solving an optimization problem using BO. The tuning procedure only requires online execution time measurement of the target loop. In order to apply BO, we model the execution time using two Gaussian process (GP) probabilistic machine learning models. Notably, we propose a locality-aware GP model, which assumes that the temporal locality effect resembles an exponentially decreasing function. By accurately modeling the temporal locality effect, our locality-aware GP model accelerates the convergence of BO. We implemented BO FSS on the GCC implementation of the OpenMP standard and evaluated its performance against other scheduling algorithms. Also, to quantify our method's performance variation on different workloads, or workload-robustness in our terms, we measure the minimax regret. According to the minimax regret, BO FSS shows more consistent performance than other algorithms. Within the considered workloads, BO FSS improves the execution time of FSS by as much as 22% and 5% on average.

show abstract

“…Policy Engine/Policies (Huck et al, 2015;Khatami, Troska, Kaiser, Ramanujam, & Serio, 2017;Laberge et al, 2019) Often, modern applications must adapt to runtime environments to ensure acceptable performance. Autonomic Performance Environment for Exascale (APEX) enables this flexibility by measuring HPX tasks, monitoring system utilization, and accepting user provided policies that are triggered by defined events.…”

mentioning

confidence: 99%

HPX - The C++ Standard Library for Parallelism and Concurrency

Kaiser¹,

Diehl²,

Lemoine³

et al. 2020

JOSS

Self Cite

View full text Add to dashboard Cite

The new challenges presented by exascale system architectures have resulted in difficulty achieving the desired scalability using traditional distributed-memory runtimes. Asynchronous many-task systems (AMT) are based on a new paradigm showing promise in addressing these challenges, providing application developers with a productive and performant approach to programming on next generation systems. HPX is a C++ Library for concurrency and parallelism that is developed by The STE||AR Group, an international group of collaborators working in the field of distributed and parallel programming (Heller, Diehl, Byerly, Biddiscombe, & Kaiser, 2017; Kaiser et al., n.d.; Tabbal, Anderson, Brodowicz, Kaiser, & Sterling, 2011). It is a runtime system written using modern C++ techniques that are linked as part of an application. HPX exposes extended services and functionalities supporting the implementation of parallel, concurrent, and distributed capabilities for applications in any domain; it has been used in scientific computing, gaming, finances, data mining, and other fields.

show abstract

Scheduling Optimization of Parallel Linear Algebra Algorithms Using Supervised Learning

Cited by 5 publications

References 22 publications

Shared memory parallelism in Modern C++ and HPX

Shared memory parallelism in Modern C++ and HPX

A Probabilistic Machine Learning Approach to Scheduling Parallel Loops with Bayesian Optimization

HPX - The C++ Standard Library for Parallelism and Concurrency

Contact Info

Product

Resources

About