Fine-Grained Allocation Algorithm for Sharing Heterogeneous Resources in Data Center

Abstract

Data in a data center are stored dispersively. The data-oriented task computing disperses big data analysis tasks to different computing nodes. The extensive use of graphics processing unit (GPU) makes it urgent and important to study how to reasonably assign heterogeneous resources to different computing frameworks. We investigate the existing big data computing framework and the GPU computing. Based on the existing cluster resource management model and the GPU management model, we propose a hybrid heterogeneous resource management model that combines CPU resources with GPU resources. The computing nodes manage local resources and implement tasks; the resource management center concertedly manage various computing frameworks. We design and implement a hybrid domain resource sharing and allocation algorithm, which allocates the hybrid domain resources to computing frameworks according to the coordinated use of them so as to fairly share the hybrid domain resources among various computing frameworks and prevent the CPU from too many tasks but the GPU or CPU from resource "hunger". The experimental results show that the allocation algorithm can increase the use of heterogeneous resources and the number of completed tasks by around 15%.