Elastic Resource Provisioning to Expand the Capacity of Cluster in Hybrid Computing Infrastructure

Abstract

Several fields of science have traditionally demanded large-scale workflow support, which requires thousands of CPU cores or more. In this paper, we investigate ways to support these scientific workflows in a heterogeneous environment in which cluster computing resources are integrating with cloud computing resources. Specifically, we first propose an architecture that utilizes cloud resources to address load balancing issues. For that, the proposed architecture measures the status of job queue on the front-end node, and then dynamically creates virtual machines from cloud pools based on the measured results to expand computing resource of the cluster. Next, we present experiment results of computational performance in hybrid infrastructure where the virtual and physical nodes are mixed.