Empirical Evaluation of a Method for Monitoring Cloud Services Based on Models at Runtime

Abstract

Cloud computing is being adopted by commercial and governmental organizations driven by the need to reduce the operational cost of their information technology resources and search for a scalable and flexible way to provide and release their software services. In this computing model, the Quality of Services (QoS) is agreed between service providers and their customers through Service Level Agreements (SLA). There is thus a need for systematic approaches with which to assess the quality of cloud services and their compliance with the SLA. In previous work, we introduced a generic method for Monitoring cloud Services using models at RunTime (MoS@RT), which allows the monitoring requirements or the metric operationalizations of these requirements to be changed at runtime without the modification of the underlying infrastructure. In this paper, we present the design of a monitoring infrastructure that supports the proposed method with its instantiation to a specific platform and reports the results of an experiment carried out to evaluate the perceived efficacy of 58 undergraduate students when using the infrastructure to configure the monitoring of cloud services deployed on the Microsoft Azure platform. The results show that the participants perceived MoS@RT to be easy to use, useful, and they also expressed their intention to use the method in the future. Although further experiments must be carried out to strengthen these results, MoS@RT has proved to be a promising monitoring method for cloud services.INDEX TERMS Cloud computing, models@runtime, quality of service (QoS), services monitoring, software as a service (SaaS).

Models@Runtime: The Development and Re-Configuration Management of Python Applications Using Formal Methods

“…They focus on component base programming rather than on a general programming language. Cedillo et al [18] described a generic method to monitor the satisfaction of non-functional requirements in cloud environments using models at runtime and SLAs. They proposed a middleware that interacts with services.…”

“…The necessity of monitoring systems through runtime models is presented in many works [3,[5][6][7][8]. Recent works, such as [6,17,18], address the problem of reconfiguring and changing the behavior of systems at runtime. Although, to date, little research has focused on providing generic tools that are independent of the application domain.…”

Continuous engineering for Industry 4.0 architectures and systems

“…In this way the architecture drivers specify not only the “what” but also the “how well” (i.e., with what degree of quality), providing clear, unambiguous input for the selection of an architecture solution for each driver. Some recent works highlight the need to evaluate scenarios and quality attribute properties at runtime, such as the approach described by Cedillo et al, 30 who suggest the use of models for monitoring and evaluating quality properties of cloud services at runtime. The recent work of Sobhy et al 29 uses time series forecasting, based on live data collected, to forecast the future performance of a system by using simulated instances of the architecture.…”

A Domain-Specific Language for Modeling IoT System Architectures That Support Monitoring

“…For the evaluation of Monitor-IoT, a quasi-experiment was used as an empirical strategy [38]. It includes the participation of undergraduate students from the last semesters of the Systems Engineering College at the University of Azuay and the University of Cuenca.…”