Improved Cloud resource allocation: how INDIGO-DataCloud is overcoming the current limitations in Cloud schedulers

Abstract

Abstract. Performing efficient resource provisioning is a fundamental aspect for any resource provider. Local Resource Management Systems (LRMS) have been used in data centers for decades in order to obtain the best usage of the resources, providing their fair usage and partitioning for the users. In contrast, current cloud schedulers are normally based on the immediate allocation of resources on a first-come, first-served basis, meaning that a request will fail if there are no resources (e.g. OpenStack) or it will be trivially queued ordered by entry time (e.g. OpenNebula). Moreover, these scheduling strategies are based on a static partitioning of the resources, meaning that existing quotas cannot be exceeded, even if there are idle resources allocated to other projects. This is a consequence of the fact that cloud instances are not associated with a maximum execution time and leads to a situation where the resources are under-utilized. These facts have been identified by the INDIGO-DataCloud project as being too simplistic for accommodating scientific workloads in an efficient way, leading to an underutilization of the resources, a non desirable situation in scientific data centers. In this work, we will present the work done in the scheduling area during the first year of the INDIGO project and the foreseen evolutions.

Orchestrating Complex Application Architectures in Heterogeneous Clouds

“…The project's design specification [26] has put the focus not only on evolving available open-source cloud components, but also on developing new solutions to cope with the project targets, introducing as a result innovative advancements at the layer of IaaS, e.g. by implementing advanced scheduling strategies based on fair share or preemptible instances [46], at the layer of PaaS, e.g. by creating SLA-based orchestration components that support deployments on multi-Clouds [76] and, finally, at the layer of SaaS, e.g.…”

An efficient cloud scheduler design supporting preemptible instances

“…Cloud operators try to solve this problem by setting resource quotas that limits the amount of resources that a user or group is able to consume by doing a static partitioning of the resources [8]. However, this kind of resource allocation automatically leads to an underutilization of the infrastructure since the partitioning needs to be conservative enough so that other users could utilize the infrastructure.…”