LB4OMP: A Dynamic Load Balancing Library for Multithreaded Applications

Abstract

Exascale computing systems will exhibit high degrees of hierarchical parallelism, with thousands of computing nodes and hundreds of cores per node. Efficiently exploiting hierarchical parallelism is challenging due to load imbalance that arises at multiple levels. OpenMP is the most widely-used standard for expressing and exploiting the ever-increasing node-level parallelism. The scheduling options in OpenMP are insufficient to address the load imbalance that arises during the execution of multithreaded applications. The limited scheduling options in OpenMP hinder research on novel scheduling techniques which require comparison with others from the literature. This work introduces LB4OMP, an open-source dynamic load balancing library that implements successful scheduling algorithms from the literature. LB4OMP is a research infrastructure designed to spur and support present and future scheduling research, for the benefit of multithreaded applications performance. Through an extensive performance analysis campaign, we assess the effectiveness and demystify the performance of all loop scheduling techniques in the library. We show that, for numerous applications-systems pairs, the scheduling techniques in LB4OMP outperform the scheduling options in OpenMP. Node-level load balancing using LB4OMP leads to reduced cross-node load imbalance and to improved MPI+OpenMP applications performance, which is critical for Exascale computing.

A Fog-Cluster Based Load-Balancing Technique

“…To further aid in moving and distributing VMs, GEP estimates the VMH load and uses this information to determine which VMs should be assigned to each of the available VMAs. An open-source dynamic load-balancing library is introduced in this study, and it incorporates effective literature-based scheduling techniques [5]. LB4OMP is a research framework that encourages and supports research programming for the benefit of multi-threaded applications.…”

Automated Scheduling Algorithm Selection and Chunk Parameter Calculation in OpenMP

“…Numerous loop scheduling algorithms were implemented in various OpenMP runtime libraries and made publicly available for OpenMP users. For instance, LB4OMP [1], an extended version of the LLVM OpenMP runtime library 2 (RTL), supports various dynamic loop scheduling (DLS) algorithms, including fixed size chunking (FSC) [20], factoring (FAC) [21], the practical variant of factoring (FAC2), tapering (TAP) [22], the practical variant of weighted factoring (WF2) [23], BOLD [24], adaptive weighted factoring (AWF), its variants (AWF-B,C,D,E) [25], adaptive factoring (AF) [26], mFAC and mAF (versions of FAC and AF with less overhead). Such a variety of many scheduling algorithms may bring users to face decision paralysis [11].…”

“…It has been shown that these three scheduling algorithms in OpenMP are insufficient for efficient scheduling of OpenMP parallel loops and that other scheduling algorithms deliver higher performance gains [1], [7], [8], [9]. However, those performance gains are achievable only via a careful (and expert) selection of scheduling algorithm and chunk parameter, on a per-loop, per-time-step, per-application, and per-system basis.…”

“…In this work, we leverage the existence of auto as a scheduling option in OpenMP and extend its implementation in the LLVM OpenMP runtime library with an expert chunk parameter (see Section 3.1) and algorithm selection methods (see Section 3.3). The automatic selection methods select a scheduling algorithm from the Auto4OMP portfolio (see Section 3.2), a subset of the portfolio of loop scheduling algorithms presented in prior work [1]. We refer to these extensions collectively as Auto4OMP.…”

“…(3) Implements the selection methods and expert chunk parameter in the LLVM OpenMP runtime library as extensions 1 to schedule(auto).…”