Automatically balancing relocatable distributed collections

Abstract

SummaryIn previous work, we introduced a distributed collections library for the APGAS for Java programming model. This library makes it possible for programmers to develop complex distributed programs thanks to the many abstractions and computation patterns supported. In particular, programmers can fully and dynamically change the distribution of data entries through high‐level abstractions. However, the problem of balancing the load between processes remains, especially in cases where multiple processes may be concurrently executing on a single host, or when the performance of the hosts used differs. To address this issue and to relieve the burden of programming a load balancing strategy for a specific application, we created a dynamic load balancer integrated into our library. This load balancer operates within a specific context in a manner which does not interfere with the program legibility. Internally, we implement a scheme inspired by the lifeline‐based global load balancer scheme first introduced in 10. We evaluate the performance of our integrated load balancer on a small‐scale Beowulf cluster.