Optimizing capacity allocation for big data applications in cloud datacenters

Spicuglia, Sebastiano; Chen, Lydia Y.; Birke, Robert; Binder, Walter

doi:10.1109/inm.2015.7140330

Cited by 9 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main goals of these scheduling algorithms are to either maximize the utilization of PMs or to maximize container performance. For example, in [14], an architecture called OptiCA was proposed in which each component of the distributed application is executed inside a container, and the containers are deployed on PMs. The main goal is to optimize the performance of big-data applications within the constraints of the available cores of the PMs.…”

Section: Placement Of Containers On Pms (C-pm)mentioning

confidence: 99%

See 1 more Smart Citation

A placement architecture for a container as a service (CaaS) in a cloud environment

2019

View full text Add to dashboard Cite

Unlike a traditional virtual machine (VM), a container is an emerging lightweight virtualization technology that operates at the operating system level to encapsulate a task and its library dependencies for execution. The Container as a Service (CaaS) strategy is gaining in popularity and is likely to become a prominent type of cloud service model. Placing container instances on virtual machine instances is a classical scheduling problem. Previous research has focused separately on either virtual machine placement on physical machines (PMs) or container, or only tasks without containerization, placement on virtual machines. However, this approach leads to underutilized or overutilized PMs as well as underutilized or overutilized VMs. Thus, there is a growing research interest in developing a container placement algorithm that considers the utilization of both instantiated VMs and used PMs simultaneously. The goal of this study is to improve resource utilization, in terms of number of CPU cores and memory size for both VMs and PMs, and to minimize the number of instantiated VMs and active PMs in a cloud environment. The proposed placement architecture employs scheduling heuristics, namely, Best Fit (BF) and Max Fit (MF), based on a fitness function that simultaneously evaluates the remaining resource waste of both PMs and VMs. In addition, another meta-heuristic placement algorithm is proposed that uses Ant Colony Optimization based on Best Fit (ACO-BF) with the proposed fitness function. Experimental results show that the proposed ACO-BF placement algorithm outperforms the BF and MF heuristics and maintains significant improvement of the resource utilization of both VMs and PMs.

show abstract

Section: Placement Of Containers On Pms (C-pm)mentioning

confidence: 99%

“…During each iteration, each ant ant k finds a placement for the set of containers C k on vm ij with a probability calculated using Eq. (14). The selection of vm ij is performed using the roulette wheel method where a random number ϵ [0, 1] is generated and a cumulative sum of the probabilities is calculated.…”

Section: Ant Colony Optimization (Aco)mentioning

confidence: 99%

A placement architecture for a container as a service (CaaS) in a cloud environment

2019

View full text Add to dashboard Cite

show abstract

“…The trace-driven simulation results cost optimized and efficient of data sharing. Spicuglia et al [10] have given the novel solution for big data cloud data allocation to enhance the capacity data storage. The method provides the robust way to store the huge data over the cloud.…”

Section: Existing Work Over Privacy and Security Issues Of Big Data mentioning

confidence: 99%

Security and Privacy Preservation on Cloud-based Big Data Analysis (CBDA): A Review

Kumar¹,

Prabhudeva²

2016

CAE

View full text Add to dashboard Cite

The term Big Data is nothing but large voluminous data, more complex data and relationship analysis of these data sets. The today's organizations, business units, government sectors, etc. are adopting the big data technique to store the enormous data generated by them. With all the significances of the big data as per economic and social benefits, lacks with many issues related to the security and privacy of the data. The modern ethnology like cloud computing will offer a scalable service for the big data with optimized cost. But the concern of privacy and security is still unsolved. This paper reflects the survey over the cloudbased big data security and privacy preservation. The survey discusses the recent work carried for privacy preservation and also existing research gap. The survey states a significant section for the future research line-up.

show abstract

“…Existing works [178,198,206] adopt models from VM-based clouds and use them to solve resource allocation problems in container-based clouds. However, the adaptation of these models is not fully compatible.…”

Section: Motivationmentioning

confidence: 99%

“…Docker Swarm allocates containers to VMs with a round-robin algorithm [206] and a Spread algorithm [2]. Spicuglia et al [198] propose an OptiCA framework that allocates containers to PM instances. Their goal is to optimize the performance of big-data applications without considering energy consumption.…”

Section: Allocation Of Containers To Pmsmentioning

confidence: 99%

An Evolutionary Computation Approach to Resource Allocation in Container-based Clouds

Tan¹

View full text Add to dashboard Cite

<p>A container-based cloud is a new trend in cloud computing that introduces more granular management of cloud resources. Compared with VM-based clouds, container-based clouds can further improve energy efficiency with a finer granularity resource allocation in data centers. The current allocation approaches for VM-based clouds cannot be used in container-based clouds. The first reason is that existing research lacks appropriate models that can represent the interaction of allocation features. Many critical features, such as VM overhead, are also not considered in the current models. The second reason is that current allocation approaches do not perform well to the three frequently encountered allocation scenarios, offline allocation, on-line allocation, and multi-objective allocation. Current approaches for these scenarios are mostly based on greedy heuristics that can be easily stuck at local optimum, or meta-heuristics that consider a simplified one-level allocation problem. Evolutionary Computation (EC) is particularly good at solving combinatorial optimization problems for both off-line and on-line scenarios. The overall goal of this thesis is to propose an EC approach to the three allocation scenarios in order to improve the performance of container-based clouds. Specifically, we aim to optimize energy consumption in all the scenarios. An additional objective, availability of the application, is considered for the multi-objective scenario. First, this thesis investigates two promising representations, vector-based and group-based. We propose two novel vector-based (e.g., Single- chromosome Genetic Algorithm (SGA) and Dual-chromosome GA (DGA)) and a group-based GA approaches for the off-line allocation scenario. Cor- responding genetic operators and decoding processes are also developed and evaluated. Two contributions have been made. Firstly, a novel offline model has been proposed based on current models with additional features. It can be used to evaluate allocation algorithms. Secondly, two types of problem representation, vector-based and group-based, are investigated and three novel approaches are proposed. The three approaches are compared with state-of-the-art approaches. The results show that all solutions produced by these approaches are better than the state-of-the-art approaches and group-based GA is the best approach. Second, this thesis proposes a novel genetic programming hyper-heuristic (GPHH) and a cooperative coevolution (CCGP)-based approach for the on-line allocation scenario. These hyper-heuristic methods can automatically generate allocation rules. For the GPHH-based approach, we develop a novel training procedure to generate reservation-based rules for allocating containers to VM instances. For the CCGP-based approach, we introduce a new terminal set and develop a training procedure to generate allocation rules for two-level allocations. We analyze both human-designed rules and generated rules to provide insights for algorithm designers. Two contributions have been proposed for the on-line problem. First, the on-line model for the on-line allocation scenario is developed. Second, a novel terminal set and training procedures are developed. The automatically generated heuristics perform significantly better than the manually designed heuristics. Third, this thesis proposes a multi-objective approach that generates a set of trade-off solutions for the cloud providers to choose from. Our novel approach, namely Nondominated Sorting-Group GA (NS-GGA), combines the group-based representation and the NSGA-II framework. The experimental results are compared with existing approaches. The results show that our proposed NS-GGA approach outperforms all other approaches. We propose two novelties. The first novelty is the multi-objective model including objectives of energy consumption and availability. The second novelty is the NS-GGA approach that combines the group-based representation with NSGA-II. The allocation solutions found by NS-GGA dominate the solutions found by other existing approaches.</p>

show abstract

Optimizing capacity allocation for big data applications in cloud datacenters

Cited by 9 publications

References 19 publications

A placement architecture for a container as a service (CaaS) in a cloud environment

A placement architecture for a container as a service (CaaS) in a cloud environment

Security and Privacy Preservation on Cloud-based Big Data Analysis (CBDA): A Review

An Evolutionary Computation Approach to Resource Allocation in Container-based Clouds

Contact Info

Product

Resources

About