Experiences with the KOALA co-allocating scheduler in multiclusters

Abstract

In multicluster systems, and more generally, in grids, jobs may require co-allocation, i.e., the simultaneous allocation of resources such as processors and input files in multiple clusters. While such jobs may have reduced runtimes because they have access to more resources, waiting for processors in multiple clusters and for the input files to become available in the right locations, may introduce inefficiencies. Moreover; as single jobs now have to rely on multiple resource managers, co-allocation introduces reliability problems. In this paper; we present two additions to the original design of our KOALA co-allocating scheduler (different priority levels ofjobs and incrementally claiming processors), and we report on our experiences with KOALA in our multicluster testbed while it was unstable.

An Efficient Protocol for Reserving Multiple Grid Resources in Advance.

“…Grid brokers relying on batch management systems at the local resources will only implement co-allocations at a high cost [9]. If the local resource manager already supports advance reservations, the Grid broker has to submit such reservations in a coordinated manner in order to allocate the requested resources.…”

An RMS Architecture for Efficiently Supporting Complex-Moldable Applications

“…Advance reservations are used as they offer a solution to the co-allocation problem [20], however, they also fragment resources [23], thus leading to inefficient resource utilisation. Redundant requests are multiple jobs targeted at individual clusters: when one of these jobs starts, the others are cancelled.…”

“…These resources are commonly managed through separate queues making it difficult to launch applications which span multiple clusters [20].…”

An Advance Reservation-Based Co-allocation Algorithm for Distributed Computers and Network Bandwidth on QoS-Guaranteed Grids

“…Therefore, each RM had better have an advance reservation capability, in order to provide a performance-guaranteed resource for a QoS-guaranteed Grid user, who also co-reserves other resources, including commercial resources. The KOALA [4] Grid scheduler and the QBETS [5] batch queue prediction service provide co-allocation of multiple cluster resources in coordination with RMs, without advance reservation, by acquiring and predicting the status of RMs. However, these strategies cannot guarantee to allocate the resources at the same time, so that the co-allocation enables efficient co-allocation of both computing and network resources provided by multiple domains, and can take administrator co-allocation options as a first step.…”