Introducing a Multithread and Multistage Mechanism for the Global Load Balancing Library of X10

Abstract

Load balancing is a major concern in massively parallel computing. X10 is a partitioned global address space language for scale-out computing and provides a global load balancing (GLB) library that shows high scalability over ten thousand CPU cores. This study proposes a multistage mechanism for GLB to assign execution stages to tasks and introduces a multithread design into GLB to allow efficient data sharing between CPU cores. The system gives high priority to tasks that are assigned to earlier stages and then proceeds with subsequent stage tasks. When a computing node runs out of tasks at the earliest stage, it requests tasks at the earliest stage from other nodes and awaits responses by processing subsequent stage tasks. When the system identifies the task termination at a certain stage, it executes a reduction operation over nodes. Programmers can define their reduction operations to gather or exchange results of completed tasks. This study provides the implementation method of the extended library and evaluates its runtime overhead using the K computer to a maximum of 256 nodes.

A self‐adjusting task granularity mechanism for the Java lifeline‐based global load balancer library on many‐core clusters

“…However, nothing is preventing us from using our library on clusters of ordinary multicore processors. In fact, this load‐balancing scheme was first designed for such systems 7 …”

“…The multithreaded global load‐balancing scheme relies on several kinds of asynchronous activities to handle the distributed computation 7 . The details of the various interactions between these activities are beyond the scope of this article.…”

“…A later evolution of this scheme, the multithreaded lifeline‐based global load balancer 7 keeps the same computation abstraction and lifeline mechanism between places but makes each place run multiple worker threads in parallel instead of a single one as per the original scheme. With this scheme, it is no longer necessary to use multiple places (or processes) per host.…”

Automatically balancing relocatable distributed collections

“…As part of the initialization process of the integrated load balancer, an initial assignment is prepared for each range of the distributed array held by the local handle. We keep these assignments in a single reserve on each host, as opposed to two in the multithreaded GLB 7 scheme, and sort them according to their priority.…”

“…In a later extension, this scheme was modified to support multiple workers per process 7,8 . In this variant, each process maintains two work queues on each host to implement work‐sharing between the workers with one queue, and work‐stealing between the host other hosts.…”