An Intelligent Approach to Resource Allocation on Heterogeneous Cloud Infrastructures

Abstract

Cloud computing systems are rapidly evolving toward multicloud architectures supported on heterogeneous hardware. Cloud service providers are widely offering different types of storage infrastructures and multi-NUMA architecture servers. Existing cloud resource allocation solutions do not comprehensively consider this heterogeneous infrastructure. In this study, we present a novel approach comprised of a hierarchical framework based on genetic programming to solve problems related to data placement and virtual machine allocation for analytics applications running on heterogeneous hardware with a variety of storage types and nonuniform memory access. Our approach optimizes data placement using the Hadoop File System on heterogeneous storage devices on multicloud systems. It guarantees the efficient allocation of virtual machines on physical machines with multiple NUMA (nonuniform memory access) domains by minimizing contention between workloads. We prove that our solutions for data placement and virtual machine allocation outperform other state-of-the-art approaches.

Integrated Data, Task and Resource Management to Speed Up Processing Small Files in Hadoop Cluster

“…Also the redistribution scheme proposed in this work has higher communication overhead. Marquez et al [6] proposed a genetic programming based data placement technique. The data placement was optimized based on reduction of data write time using genetic algorithm.…”

Scalability and Reliability Analysis of a Novel Cloud Platform for TSO-DSO Information and Data Exchange

“…HDFS is one of the most widely used file systems. Similarly, most cloud providers offer non-uniformed memory architecture nodes that can be leveraged to optimize virtual machine allocation [13]. Few efforts have recently been made to address the issues of data placement on heterogeneous storage infrastructure [14], virtual machine allocation on various servers [15], and intelligent resource allocation on the cloud [16], [17].…”