Adaptive Task Partitioning for Performance Evaluation in Cluster based Heterogeneous Environments

Abstract

Due to continual server replacement, datacenter-scale clusters are developing toward heterogeneous hardware designs. Meanwhile, datacenters are frequently used by a variety of users for a variety of purposes. Due to multi-tenant interferences, it frequently exhibits high performance heterogeneity. When contrasted to in-house dedicated clusters, deploying MapReduce on such heterogeneous clusters poses major hurdles in attaining adequate application performance. Heterogeneity can cause significant performance degradation in job execution, despite current optimizations on task scheduling and load balancing, because most MapReduce implementations were developed for homogeneous contexts. To make scheduling decisions, the majority of extant adaptive strategies assume a priori knowledge of particular job characteristics. However, without spending a significant cost, such information is not readily available. The suggested framework Adaptive Control Self-tuning provides a significant improvement over existing methods at moderate to high system utilizations, according to the evaluation results.