An Overview on Quality Evaluation of Self-Adaptive Systems

Raibulet, Claudia; Fontana, Francesca Arcelli; Capilla, Rafael; Canal, Carlos

doi:10.1016/b978-0-12-802855-1.00013-7

Cited by 18 publications

(11 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Raibulet et al has recently presented an overview of some existing approaches for the evaluation of self-adaptive systems. 13 According to their analysis on the published works, when quality attributes (such as performance and reliability) are considered, these attributes are evaluated at runtime in 95% of the cases, and at design time in 5% of them. The authors also pointed out that none of these approaches associated these evaluations to a tool, which may hamper the evaluative part.…”

Section: Motivation and Challengesmentioning

confidence: 99%

A procedural and flexible approach for specification, modeling, definition, and analysis for self‐adaptive systems

2021

View full text Add to dashboard Cite

An adaptive system can modify its settings at runtime as a response to changes in its operational environment. To analyse this kind of systems at design time is a difficult task since it requires considering the system together with the adaptation operations, and taking into account how such adaptations act on the system. In order to use simulation-based techniques for the analysis of such systems, we not only need precise executable models of the systems to be analyzed, but also to capture the semantics of their adaptation mechanisms. Given the wide range and flexibility of adaptation operations, we need ways to allow the definition of new operations. We present a flexible approach for the definition and simulation-based analysis in design-time of adaptive component-based systems. Our approach combines an extension of the Palladio component model in e-Motions, a model of the adaptation mechanisms, and elastic requirements specification using the SYBL language. From the model of the system, its adaptation mechanisms, and its requirements, an executable Maude specification is generated for simulation. The application of the approach is illustrated on a use case that comprises some components and adaptations rules. The example is then analyzed using simulations. It is also shown that it is indeed possible to define additional metrics, specify adaptation requirements and rules which conduct simulations of the models in a more flexible way, and that the results of the simulation performed from these definitions can be used to carry on a valuable predictive performance analysis. K E Y W O R D Sadaptation specification, design-time analysis, modeling, self-adaptive behavior, self-adaptive models INTRODUCTIONSoftware architecture has become an essential element in building consistent systems as the size and complexity of software systems have increased. The necessity to take into account not only the overall structure of the system itself but also the allocation, functionality, and communication of the software components became a challenge for building

show abstract

Section: Motivation and Challengesmentioning

confidence: 99%

A procedural and flexible approach for specification, modeling, definition, and analysis for self‐adaptive systems

2021

View full text Add to dashboard Cite

show abstract

“…For example, an SAS can be evaluated based on its failure density, cost, or time to adapt. Raibulet et al proposed a taxonomy for evaluating SAS [108] which includes evaluation scope, evaluation time, evaluation mechanism, evaluation perspective, and evaluation type. The study proposed by Chen et.al [18] shows that using a genetic algorithm for predicting Quality of Service (QoS) provided higher accuracy results.…”

Section: Related Workmentioning

confidence: 99%

The Application of Machine Learning in Self-Adaptive Systems: A Systematic Literature Review

Saputri

Lee

2020

IEEE Access

View full text Add to dashboard Cite

Self-adaptive systems have been studied in software engineering over the past few decades attempting to address challenges within the field. There is a continuous significant need to fully understand the behavior and characteristics of the systems that operate in dynamic environments. By learning the behavior pattern of the environment, we can avoid unnecessary adaptations imbalance efforts for adaptation. As such, there exist research in the area of machine learning aimed at understanding dynamic environments regarding self-adaptive systems. Objective: This study aims to help software practitioners to address adaptation concerns by performing a systematic literature review that provides a comprehensive overview of using machine learning (ML) in self-adaptive systems. We summarize state-ofthe-art Of the ML approaches used to handle self-adaptation to help software engineers in the proper selection of ML techniques based on the adaptation concern. Method: This review examines research published between 2001 and 2019 on ML implementation in self-adaptive systems, focusing on the adaptation aspects and purposes. The review was conducted by analyzing major scientific databases that resulted in 78 primary studies from 315 papers from an automatic search. Result: Finally, this study recommends three future research directions to enhance the application of machine learning in self-adaptive systems.

show abstract

“…While playing back a recording of a real failure trace as input for a simulated system improves the credibility of the simulation-based experiments, the resulting output of a single recorded real failure trace lacks generality. Employing a single recorded real failure trace as input for SHS evaluation only supports single experiment run and does not support any claim on certain qualitative evaluation metrics, such as resilience, reliability, and robustness testing [3]. While such a trace contains realistic characteristics of occurrences of failures, it only captures one possible future for the simulated SHS and fails to cover a large and representative spectrum of the input space.…”

Section: Recorded Real Failure Tracementioning

confidence: 99%

“…The system is also subject to changes in unforeseen ways as a result of the adaptation [2]. On the other hand, these systems are designed to be operated in highly dynamic environments, and therefore require continuous monitoring of their behavior and execution environment [3].…”

Section: Introductionmentioning

confidence: 99%

“…Over the last few decades, a huge body of work on engineering SAS including SHS has been developed by researchers and engineers. As distinguished in [3], during the lifetime of a SAS in general, evaluation concerns design-time, deployment-time, and runtime decisions. As a first step in evaluating a SAS, the quality of the employed self-adaptive approaches is investigated.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Self-Healing Systems: An Analysis of the State-of-the-Art and Required Improvements

Ghahremani

Giese

2020

Computers

View full text Add to dashboard Cite

Evaluating the performance of self-adaptive systems is challenging due to their interactions with often highly dynamic environments. In the specific case of self-healing systems, the performance evaluations of self-healing approaches and their parameter tuning rely on the considered characteristics of failure occurrences and the resulting interactions with the self-healing actions. In this paper, we first study the state-of-the-art for evaluating the performances of self-healing systems by means of a systematic literature review. We provide a classification of different input types for such systems and analyse the limitations of each input type. A main finding is that the employed inputs are often not sophisticated regarding the considered characteristics for failure occurrences. To further study the impact of the identified limitations, we present experiments demonstrating that wrong assumptions regarding the characteristics of the failure occurrences can result in large performance prediction errors, disadvantageous design-time decisions concerning the selection of alternative self-healing approaches, and disadvantageous deployment-time decisions concerning parameter tuning. Furthermore, the experiments indicate that employing multiple alternative input characteristics can help with reducing the risk of premature disadvantageous design-time decisions.

show abstract

An Overview on Quality Evaluation of Self-Adaptive Systems

Cited by 18 publications

References 10 publications

A procedural and flexible approach for specification, modeling, definition, and analysis for self‐adaptive systems

A procedural and flexible approach for specification, modeling, definition, and analysis for self‐adaptive systems

The Application of Machine Learning in Self-Adaptive Systems: A Systematic Literature Review

Evaluation of Self-Healing Systems: An Analysis of the State-of-the-Art and Required Improvements

Contact Info

Product

Resources

About