Locality-Aware Work Stealing Based on Online Profiling and Auto-Tuning for Multisocket Multicore Architectures

Abstract

Modern mainstream powerful computers adopt multisocket multicore CPU architecture and NUMA-based memory architecture. While traditional work-stealing schedulers are designed for single-socket architectures, they incur severe shared cache misses and remote memory accesses in these computers. To solve the problem, we propose a locality-aware work-stealing (LAWS) scheduler, which better utilizes both the shared cache and the memory system. In LAWS, a load-balanced task allocator is used to evenly split and store the dataset of a program to all the memory nodes and allocate a task to the socket where the local memory node stores its data for reducing remote memory accesses. Then, an adaptive DAG packer adopts an auto-tuning approach to optimally pack an execution DAG into cache-friendly subtrees. After cache-friendly subtrees are created, every socket executes cache-friendly subtrees sequentially for optimizing shared cache usage. Meanwhile, a triple-level work-stealing scheduler is applied to schedule the subtrees and the tasks in each subtree. Through theoretical analysis, we show that LAWS has comparable time and space bounds compared with traditional work-stealing schedulers. Experimental results show that LAWS can improve the performance of memory-bound programs up to 54.2% on AMD-based experimental platforms and up to 48.6% on Intel-based experimental platforms compared with traditional work-stealing schedulers. ACM Reference Format:Quan Chen and Minyi Guo. 2015. Locality-aware work stealing based on online profiling and auto-tuning for multisocket multicore architectures. ACM Trans. " , which was published in the International Conference on Supercomputing (ICS 2014). The 30% new material comes from two aspects:-We have analyzed the theoretical time and space bounds of LAWS. Based on our analysis, the theoretical time and space bounds are comparable to the original random work-stealing scheduler. -This article has also significantly enhanced the experimental evaluation. We have evaluated the performance of LAWS on both Intel-based platforms and AMD-based platforms.

Bandwidth and Locality Aware Task-stealing for Manycore Architectures with Bandwidth-Asymmetric Memory

“…Targeting scheduling systems for task-based programs, a large amount of prior work aims to improve energy-efficiency [38,41], to improve data locality [9,10], or to reduce scheduling overhead [17,29]. However, with the increasing bandwidth requirements of computing tasks, many papers have also conducted related research for efficient bandwidth usage.…”

“…Many task-stealing schedulers have been proposed to improve data locality by reducing shared cache misses [10,11] and increasing local memory accesses [9,25,40,46]. Based on Charm++ [19], NUMALB [32] is proposed to balance the workload while avoiding unnecessary migrations and reducing cross-core communication.…”

“…However, users have to give the level manually in HWS or provide a number of command line arguments for the scheduler to calculate the boundary level in CAB. To relieve the above burden, CATS [10] was proposed to divide task graph based on the online information, without extra user-provided information. These techniques assume that the data accessed by a task is known by analyzing the task graph, which is not always true in real-system applications.…”

“…Those efforts are orthogonal to BATS. [10] HWS [34] DFA [15] LAWS [9] RELWS [25] HPT [52] Jenga [45] There is also prior work on improving the performance of other schedulers, such as OpenMP. Olivier et al [30] proposed a hierarchical scheduling strategy that uses one thread to steal work on behalf of all of the threads in a chip.…”

Dense Matrix Computations on NUMA Architectures with Distance-Aware Work Stealing

“…Recently, LAWS [12] proposes a runtime library for Divide and Conquer applications in NUMA systems. It features a work stealing algorithm designed for NUMA systems, very focused in reducing remote memory accesses and last-level cache pollution.…”