OptiSpot: minimizing application deployment cost using spot cloud resources

Dubois, Daniel J.; Casale, Giuliano

doi:10.1007/s10586-016-0568-7

Cited by 32 publications

(13 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We propose queueing network (QN) models and stochastic well formed nets (SWNs) of incremental complexity and accuracy able to estimate MapReduce job execution times for multiple users and under unreliable resources. In particular, we analyze a Cloud-based scenario where the cluster, to save execution costs, includes also spot virtual machines (VMs) [14]. The utilization of spot VMs offers large discounts in VM prices, with the drawback of a nonguaranteed availability level.…”

Section: Introductionmentioning

confidence: 99%

Modeling Performance of Hadoop Applications: A Journey from Queueing Networks to Stochastic Well Formed Nets

Ardagna

Bernardi

Gianniti

et al. 2016

Algorithms and Architectures for Parallel Processing

View full text Add to dashboard Cite

Nowadays, many enterprises commit to the extraction of actionable knowledge from huge datasets as part of their core business activities. Applications belong to very different domains such as fraud detection or one-to-one marketing, and encompass business analytics and support to decision making in both private and public sectors. In these scenarios, a central place is held by the MapReduce framework and in particular its open source implementation, Apache Hadoop. In such environments, new challenges arise in the area of jobs performance prediction, with the needs to provide Service Level Agreement guarantees to the end-user and to avoid waste of computational resources. In this paper we provide performance analysis models to estimate MapReduce job execution times in * Acknowledgments: This work has received funding from the European Union Horizon 2020 research and innovation program under grant agreement No. 644869 (DICE). Experimental data are available as open data at https://zenodo.org/record/58847#.V5i0wmXA45Q. 1Hadoop clusters governed by the YARN Capacity Scheduler. We propose models of increasing complexity and accuracy, ranging from queueing networks to stochastic well formed nets, able to estimate job performance under a number of scenarios of interest, including also unreliable resources. The accuracy of our models is evaluated by considering the TPC-DS industry benchmark running experiments on Amazon EC2 and the CINECA Italian supercomputing center. The results have shown that the average accuracy we can achieve is in the range 9-14%.

show abstract

Section: Introductionmentioning

confidence: 99%

Modeling Performance of Hadoop Applications: A Journey from Queueing Networks to Stochastic Well Formed Nets

Ardagna

Bernardi

Gianniti

et al. 2016

Algorithms and Architectures for Parallel Processing

View full text Add to dashboard Cite

show abstract

“…This arrival rate usually does not depend on the system state, that is, it does not depend on the number of other tenants in the system [41]. The open multiclass queuing network model is a widely used technique to optimize the deployment of services in cloud computing environments [54,55]. This study is a further contribution to literature by applying an open multiclass queuing model to provide optimal solutions for deploying cloud services to guarantee the required degree of multitenancy isolation.…”

Section: Open Multiclass Queuing Network Modelmentioning

confidence: 99%

Optimal deployment of components of cloud-hosted application for guaranteeing multitenancy isolation

2019

View full text Add to dashboard Cite

One of the challenges of deploying multitenant cloud-hosted services that are designed to use (or be integrated with) several components is how to implement the required degree of isolation between the components when there is a change in the workload. Achieving the highest degree of isolation implies deploying a component exclusively for one tenant; which leads to high resource consumption and running cost per component. A low degree of isolation allows sharing of resources which could possibly reduce cost, but with known limitations of performance and security interference. This paper presents a model-based algorithm together with four variants of a metaheuristic that can be used with it, to provide near-optimal solutions for deploying components of a cloud-hosted application in a way that guarantees multitenancy isolation. When the workload changes, the model-based algorithm solves an open multiclass QN model to determine the average number of requests that can access the components and then uses a metaheuristic to provide near-optimal solutions for deploying the components. Performance evaluation showed that the obtained solutions had low variability and percent deviation when compared to the reference/optimal solution. We also provide recommendations and best practice guidelines for deploying components in a way that guarantees the required degree of isolation.

show abstract

“…Finally, some research jointly considers pricing models and the variable resource availability [3,10,28], and self-organizing autoscaling techniques [10] to optimize the load allocation. In our previous work [8,9] we applied a heuristic-based approach to find the cheapest cloud resources for the application deployment while taking into account the variable availability that is typical of spot and preemptible instances. However, to the best of our knowledge, none of the existing approaches proposes the application refactoring as support for the automated resource provisioning.…”

Section: Related Workmentioning

confidence: 99%

“…In our previous work [8,9] we have introduced an approach to minimize the costs of cloud resources while optimizing the allocation of software components. Instead, in this work we additionally consider model-driven application refactorings and we evaluate their impact in the process of minimizing cloud resources costs.…”

Section: Introductionmentioning

confidence: 99%

Model-Driven Application Refactoring to Minimize Deployment Costs in Preemptible Cloud Resources

Dubois

Trubiani

Casale

2016

2016 IEEE 9th International Conference on Cloud Computing (CLOUD)

Self Cite

View full text Add to dashboard Cite

Abstract-Performance assessment of cloud-based applications requires new methodologies to deal with the complexity of software systems and the variability of cloud resources. In this paper, we address the problem of reducing the total costs for running cloud-based applications while fulfilling servicelevel objectives (SLOs). To this end, we define an approach to refactor a cloud application in such a way that, when it is deployed, it requires less computational capacity and therefore less resources. We experimented our approach on top of a modified optimal provisioning heuristic designed for preemptible cloud resources and the results show that it reduces deployment costs, up to 60% when compared to the same approach, but without model-driven application refactoring.Keywords-software performance engineering; cloud computing; application refactoring; resource provisioning. I. INTRODUCTIONCloud computing has become popular in organizations as a mean to outsource part of their IT infrastructures and lower their costs. As the number of cloud providers and offerings are increasing, it is becoming more challenging for organizations to make decisions on which cloud resources to rent for deploying their applications. Cloud resources are not only characterized by different performance in terms of processors, memory, etc., but also by different pricing models that result in different levels of reliability. For example, Amazon EC2 spot instances and Google CE's preemptible VMs are rented at a cheaper price than on-demand resources, but they carry the continuous risk of being claimed back by the cloud provider. While these types of resources operate at lower costs than others, they introduce new issues since they can become unavailable at any time.Model-driven approaches stem from the idea of creating and quantitatively analyzing performance models from high-level specifications. Example of languages are UML diagrams [6] and Palladio [2,19], which can be translated into performance models that are easier to analyze such as queueing networks (QN) [1]. Based on the generated performance models, model-driven approaches optimally refactor [18] the application architecture with the purpose of meeting SLOs [5,26].The focus of this paper is on the performance evaluation of cloud-based applications. We aim to exploit predictive performance results and provide feedback while considering two main sources of information: (i) software knowledge, i.e., by experimenting design alternatives that optimize the application model, and (ii) resource knowledge, i.e., by selecting cloud resources that minimize the deployment costs. To this end, the adoption of software refactorings is aimed at improving the system performance at the application level, when reasoning on predictive quantitative results may avoid

show abstract

OptiSpot: minimizing application deployment cost using spot cloud resources

Cited by 32 publications

References 31 publications

Modeling Performance of Hadoop Applications: A Journey from Queueing Networks to Stochastic Well Formed Nets

Modeling Performance of Hadoop Applications: A Journey from Queueing Networks to Stochastic Well Formed Nets

Optimal deployment of components of cloud-hosted application for guaranteeing multitenancy isolation

Model-Driven Application Refactoring to Minimize Deployment Costs in Preemptible Cloud Resources

Contact Info

Product

Resources

About