Multi-dimensional dynamic loop scheduling algorithms

Abstract

Abstract-Distributed Computing Systems are a viable and less expensive alternative to parallel computers. However, a serious difficulty in concurrent programming of a distributed system is how to deal with scheduling and load balancing of such a system which may consist of heterogeneous computers. Loop scheduling schemes for parallel computers and computer clusters have been proposed in the past. All these schemes are one-dimensional because they partition only the outermost loop of a nested loop construct. In this work, we consider scheduling nested loops with many dimensions. We propose a new methodology which partitions many levels (or dimensions) of nested loops. These new schemes show superior performance over the existing schemes. We implement our new schemes on a network of computers and make performance comparisons with other existing schemes. We expect the new schemes to be particularly useful for multi-core systems because of the fine granularity of the generated tasks.

Scalable Loop Self-Scheduling Schemes for Large-Scale Clusters and Cloud Systems

“…In [10,22,23], the heterogeneity of different cluster systems was considered, in order to get better load balancing. Multi-dimensional loop scheduling schemes were studied in [24,25].…”

“…Thus, it can cause load imbalance in the parallel computation. The adjoint convolution has a size of 800 × 800 and the arrays are generated randomly.Unlike TSS, the FSS and GSS, are known to have a number of their (last) scheduling steps of chunk size = 1 ([25]). This number of steps equals the number of workers, which is considerable for massive number of workers.…”