CloudsStorm: A framework for seamlessly programming and controlling virtual infrastructure functions during the DevOps lifecycle of cloud applications

Abstract

The infrastructure-as-a-service (IaaS) model of cloud computing provides virtual infrastructure functions (VIFs), which allow application developers to flexibly provision suitable virtual machines' (VM) types and locations, and even configure the network connection for each VM. Because of the pay-as-you-go business model, IaaS provides an elastic way to operate applications on demand. However, in current cloud applications DevOps (software development and operations) lifecycle, the VM provisioning steps mainly rely on manually leveraging these VIFs. Moreover, these functions cannot be programmatically embedded into the application logic to control the infrastructure at runtime. Especially, the vendor lock-in issue, which different clouds provide different VIFs, also enlarges this gap between the cloud infrastructure management and application operation. To mitigate this gap, we designed and implemented a framework, CloudsStorm, which enables developers to easily leverage VIFs of different clouds and program them into their cloud applications. To be specific, CloudsStorm empowers applications with infrastructure programmability at design-level, infrastructure-level, and application-level. CloudsStorm also provides two infrastructure controlling modes, ie, active and passive mode, for applications at runtime. Besides, case studies about operating task-based and big data applications on clouds show that the monetary cost is significantly reduced through the seamless and on-demand infrastructure management provided by CloudsStorm. Finally, the scaling and recovery operation evaluations of CloudsStorm are performed to show its controlling performance. Compared with other tools, ie, "jcloud" and "cloudinit.d", the scaling and provisioning performance evaluations demonstrate that CloudsStorm can achieve at least 10% efficiency improvement in our experiment settings. KEYWORDS DevOps, federated clouds, infrastructure-as-a-service, networked virtual infrastructureThis is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. each individual VM instead of the entire infrastructure topology. In comparison, there are also some environment-centric 1 tools to help developers orchestrate their applications, which include Puppet, # Chef, ‖ etc. These tools more focus on the cluster management, ie, deployment and configuration, instead of provisioning and scaling automation. Some academic research studies also propose architectures for developing and orchestrating applications on clouds, eg, CometCloud 6 and mOSAIC. 7 However, most of these architectures themselves are platforms, which require manually to set up the cluster in advance and lack the ability of provisioning VM resources. (3) Application-level. Hadoop applications require application-level programmability to directly adjust the infrastructure to fit for the application constraints or workload. Because of the pay-as-you-go business ...

Data Processing and Analytics for Data-Centric Sciences

“…There is a need to develop data processing technologies that address the problem by abstracting from (and virtualising) the platform(s) that take care of executing the processing pipeline. Such technologies should go in tandem with optimisation technologies and should provide the data processing designer with fine-grained processing directives and facilitate detailed specification of processing algorithms [35].…”

ALLSTAR: A Blockchain Based Decentralized Ecosystem for Cloud and Edge Computing

“…All these works lack an efficient solution for the application to program and control the computing infrastructure in the DevOps lifecycle. Although our previous work CloudsStorm [7], [8] framework tackling the resource management issue from the application DevOps perspective, it still mainly focuses on working with Cloud resources, which is insufficient for a more complex Edge computing environment.…”

A Trustworthy Blockchain-based Decentralised Resource Management System in the Cloud

“…It bridges the gap between real-time scheduling theory and Xen, and provides a platform for integrating a broad range of realtime and embedded systems. The performance guarantee at the software system level relies on the optimisation between application and infrastructure, e.g., CloudStorm [17] for service applications and PrEstoCloud [18] for real-time Big Data using an extension of the Fog computing paradigm to the extreme Edge of the network, real-time container scheduling [19]. However, the engineering of quality critical applications across heteronomous infrastructure for DApps is still very challenging, due to diverse programming interfaces, and the lack of effective programming methods.…”