CMI: An online multi-objective genetic autoscaler for scientific and engineering workflows in cloud infrastructures with unreliable virtual machines

Monge, David A.; Pacini, Elina; Mateos, Cristian; Alba, Enrique; Garino, Carlos García

doi:10.1016/j.jnca.2019.102464

Cited by 17 publications

(8 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The majority of resource provision techniques are guided by QoS constraints defined by users. There are two basic constraints, i.e., the deadline [26], [24], [22], [21], [18], [16] and the monetary budget [25], [23], [15], [14]. While the deadline constraint is usually satisfied, unreliable resources are used to the maximum extent to limit the monetary costs.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Fuzzy Logic Controller for Tasks Scheduling Using Unreliable Cloud Resources

Oikonomou,

Kolomvatsos,

Tziritas

et al. 2020

Preprint

View full text Add to dashboard Cite

The Cloud infrastructure offers to end users a broad set of heterogenous computational resources using the pay-asyou-go model. These virtualized resources can be provisioned using different pricing models like the unreliable model where resources are provided at a fraction of the cost but with no guarantee for an uninterrupted processing. However, the enormous gamut of opportunities comes with a great caveat as resource management and scheduling decisions are increasingly complicated. Moreover, the presented uncertainty in optimally selecting resources has also a negatively impact on the quality of solutions delivered by scheduling algorithms. In this paper, we present a dynamic scheduling algorithm (i.e., the Uncertainty-Driven Scheduling -UDS algorithm) for the management of scientific workflows in Cloud. Our model minimizes both the makespan and the monetary cost by dynamically selecting reliable or unreliable virtualized resources. For covering the uncertainty in decision making, we adopt a Fuzzy Logic Controller (FLC) to derive the pricing model of the resources that will host every task. We evaluate the performance of the proposed algorithm using real workflow applications being tested under the assumption of different probabilities regarding the revocation of unreliable resources. Numerical results depict the performance of the proposed approach and a comparative assessment reveals the position of the paper in the relevant literature.

show abstract

Section: Related Workmentioning

confidence: 99%

“…High spot prices defined by users trigger the switch from unreliable to reliable resources. In [25], we can find a discussion on the problem of auto-scaling public resources using a Multi-objective Genetic Algorithm (GA). The makespan, the monetary cost and Outof-Bid (OOB) errors are considered as the targets of the minimization process.…”

Section: Related Workmentioning

confidence: 99%

A Fuzzy Logic Controller for Tasks Scheduling Using Unreliable Cloud Resources

Oikonomou,

Kolomvatsos,

Tziritas

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Both subproblems are NP-hard and, therefore, the solutions proposed to date are mainly based on heuristics [39,7,40,41].…”

Section: Workflow Autoscaling In Cloudsmentioning

confidence: 99%

Reinforcement Learning-based Application Autoscaling in the Cloud: A Survey

Garí¹,

Monge²,

Pacini³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Reinforcement Learning (RL) has demonstrated a great potential for automatically solving decision making problems in complex uncertain environments. Basically, RL proposes a computational approach that allows learning through interaction in an environment of stochastic behavior, with agents taking actions to maximize some cumulative short-term and long-term rewards. Some of the most impressive results have been shown in Game Theory where agents exhibited super-human performance in games like Go or Starcraft 2, which led to its adoption in many other domains including Cloud Computing. Particularly, workflow autoscaling exploits the Cloud elasticity to optimize the execution of workflows according to a given optimization criteria. This is a decision-making problem in which it is necessary to establish when and how to scale-up/down computational resources; and how to assign them to the upcoming processing workload. Such actions have to be taken considering some optimization criteria in the Cloud, a dynamic and uncertain environment. Motivated by this, many works apply RL to the autoscaling problem in Cloud. In this work we survey exhaustively those proposals from major venues, and uniformly compare them based on a set of proposed taxonomies. We also discuss open problems and provide a prospective of future research in the area.

show abstract

“…As an elastic, reliable, dynamic services provider, Cloud computing provisions computing resources on the basis of CPU (Central Processing Unit) [3,12,13], RAM (Random Access Memory) [14,15], GPU (Graphics Processing Unit) [16,17], Disk Capacity [3,13] and Network Bandwidth [14,18]. From another perspective, "time", that the whole service life cycle of Cloud computing platform, and "space", that the real physical place to emplace Cloud computing physical devices, are also two pivotal resources of Cloud computing.…”

Section: Introductionmentioning

confidence: 99%

“…Consequently, how to schedule components of Cloud computing in an efficient, energy-saving, balanced method, is proved to be a critical factor influencing the orientation of Cloud computing in society. Whereas, the huge scale of Cloud computing, the complexity of scenarios, the unpredictability of user requests, the randomness of electronic components, and uncertain temperature of various components presented in the running process, pose challenges to efficient and effective scheduling of Cloud computing in the emerging trend [6,12,15,[18][19][20][21][22]. To lead more precise comprehension of Cloud computing, we will briefly review the development history of Cloud and its scheduling approaches.…”

Section: Introductionmentioning

confidence: 99%

Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions

Zhou¹,

Tian²,

Buyya³

2021

Preprint

View full text Add to dashboard Cite

As the quantity and complexity of information processed by software systems increase, large-scale software systems have an increasing requirement for high-performance distributed computing systems. With the acceleration of the Internet in Web 2.0, Cloud computing as a paradigm to provide dynamic, uncertain and elastic services has shown superiorities to meet the computing needs dynamically. Without an appropriate scheduling approach, extensive Cloud computing may cause high energy consumptions and high cost, in addition that high energy consumption will cause massive carbon dioxide emissions. Moreover, inappropriate scheduling will reduce the service life of physical devices as well as increase response time to users' request. Hence, efficient scheduling of resource or optimal allocation of request, that usually a NP-hard problem, is one of the prominent issues in emerging trends of Cloud computing. Focusing on improving quality of service (QoS), reducing cost and abating contamination, researchers have conducted extensive work on resource scheduling problems of Cloud computing over years. Nevertheless, growing complexity of Cloud computing, that the super-massive distributed system, is limiting the application of scheduling approaches. Machine learning, a utility method to tackle problems in complex scenes, is used to resolve the resource scheduling of Cloud computing as an innovative idea in recent years. Deep reinforcement learning (DRL), a combination of deep learning (DL) and reinforcement learning (RL), is one branch of the machine learning and has a considerable prospect in resource scheduling of Cloud computing. This paper surveys the methods of resource scheduling with focus on DRL-based scheduling approaches in Cloud computing, also reviews the application of DRL as well as discusses challenges and future directions of DRL in scheduling of Cloud computing.

show abstract

CMI: An online multi-objective genetic autoscaler for scientific and engineering workflows in cloud infrastructures with unreliable virtual machines

Cited by 17 publications

References 39 publications

A Fuzzy Logic Controller for Tasks Scheduling Using Unreliable Cloud Resources

A Fuzzy Logic Controller for Tasks Scheduling Using Unreliable Cloud Resources

Reinforcement Learning-based Application Autoscaling in the Cloud: A Survey

Deep Reinforcement Learning-based Methods for Resource Scheduling in Cloud Computing: A Review and Future Directions

Contact Info

Product

Resources

About