Training Prediction Models for Rule-Based Self-Adaptive Systems

Ghahremani, Sona; Adriano, Christian M.; Giese, Holger

doi:10.1109/icac.2018.00031

Cited by 16 publications

(14 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, constructing a utility function in such a way is challenging due to various sources of uncertainty, such as non-linearities, complex dynamic architectures, and blackbox models. To address this issue, we train prediction models for the utility of systems to replace the manually and analytically defined utility functions [Ghahremani et al 2018], and we want to study how such prediction models can be integrated into our scheme to learn and evolve utility functions online for dynamic architectures. Finally, we want to investigate the concurrent execution of adaptation rules and to broaden the spectrum of self-adaptive systems to which our scheme could be applied by studying other systems than mRUBiS and other self-* properties than self-healing.…”

Section: Discussionmentioning

confidence: 99%

Improving Scalability and Reward of Utility-Driven Self-Healing for Large Dynamic Architectures

Ghahremani

Giese

Vogel

2019

ACM Trans. Auton. Adapt. Syst.

Self Cite

View full text Add to dashboard Cite

Self-adaptation can be realized in various ways. Rule-based approaches prescribe the adaptation to be executed if the system or environment satisfies certain conditions. They result in scalable solutions but often with merely satisfying adaptation decisions. In contrast, utility-driven approaches determine optimal decisions by using an often costly optimization, which typically does not scale for large problems. We propose a rule-based and utility-driven adaptation scheme that achieves the benefits of both directions such that the adaptation decisions are optimal, whereas the computation scales by avoiding an expensive optimization. We use this adaptation scheme for architecture-based self-healing of large software systems. For this purpose, we define the utility for large dynamic architectures of such systems based on patterns that define issues the self-healing must address. Moreover, we use pattern-based adaptation rules to resolve these issues. Using a pattern-based scheme to define the utility and adaptation rules allows us to compute the impact of each rule application on the overall utility and to realize an incremental and efficient utility-driven self-healing. In addition to formally analyzing the computational effort and optimality of the proposed scheme, we thoroughly demonstrate its scalability and optimality in terms of reward in comparative experiments with a static rule-based approach as a baseline and a utility-driven approach using a constraint solver. These experiments are based on different failure profiles derived from real-world failure logs. We also investigate the impact of different failure profile characteristics on the scalability and reward to evaluate the robustness of the different approaches. S. Ghahremani et al.approaches Kephart and Das 2007] combine both phases. Adaptation is executed for specific events and under specific conditions by adaptation rules. In such approaches, events trigger the rules that subsequently check their conditions. If the conditions are fulfilled, the actions of the rules are applied and result in the envisioned changes. Thus, the applicable rules are identified (matched) and executed to adapt the system configuration at runtime. The main strengths of such approaches are the readability, elegance, and the efficient processing of the rules. The drawbacks are the fact that the adaptation decisions are often only satisfying and the limited expressiveness of rules since rules typically just relate events to actions [Fleurey and Solberg 2009] without defining and performing any further computation for analysis and planning (e.g., to identify optimal actions). On the other hand, utility-driven approaches [Esfahani et al. 2013;Kephart and Walsh 2004] determine optimal adaptation decisions by using optimization techniques for planning that are guided by a utility function. A utility function determines how valuable each possible system configuration is, and the optimization aims at identifying optimal configurations. However, the optimization usually prevents the ...

show abstract

Section: Discussionmentioning

confidence: 99%

Improving Scalability and Reward of Utility-Driven Self-Healing for Large Dynamic Architectures

Ghahremani

Giese

Vogel

2019

ACM Trans. Auton. Adapt. Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Another notable effort of retrained modeling based on the Decision Tree (DT) family (e.g., M5 decision tree [29]), such as FUSION [19] and Guo et al [20], where the performance model is discarded and rebuilt using all the available data when the adaptable software collects new information. A general framework for modeling performance of adaptable software using the retrained method, which is agnostic to the learning algorithm, were proposed by Ghahremani et al [30].…”

Section: Prior Retrained Performance Modelingmentioning

confidence: 99%

All Versus One: An Empirical Comparison on Retrained and Incremental Machine Learning for Modeling Performance of Adaptable Software

Chen

2019

2019 IEEE/ACM 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)

View full text Add to dashboard Cite

Given the ever-increasing complexity of adaptable software systems and their commonly hidden internal information (e.g., software runs in the public cloud), machine learning based performance modeling has gained momentum for evaluating, understanding and predicting software performance, which facilitates better informed self-adaptations. As performance data accumulates during the run of the software, updating the performance models becomes necessary. To this end, there are two conventional modeling methods: the retrained modeling that always discard the old model and retrain a new one using all available data; or the incremental modeling that retains the existing model and tunes it using one newly arrival data sample. Generally, literature on machine learning based performance modeling for adaptable software chooses either of those methods according to a general belief, but they provide insufficient evidences or references to justify their choice. This paper is the first to report on a comprehensive empirical study that examines both modeling methods under distinct domains of adaptable software, 5 performance indicators, 8 learning algorithms and settings, covering a total of 1,360 different conditions. Our findings challenge the general belief, which is shown to be only partially correct, and reveal some of the important, statistically significant factors that are often overlooked in existing work, providing evidence-based insights on the choice.

show abstract

“…To address these threats, we used mRUBiS as our simulation environment, which has been accepted as an exemplar by the research community on self-adaptive software and has been extensively tested by students in the scope of four courses on self-adaptive software. The soundness of the employed self-healing approaches has been confirmed in our previous works [43,54,58]. Since the employed utility functions are the same for all the considered self-healing approaches, the specific way of constructing these utility functions is not a threat to the validity of the experiments, as none of the claims depend on the specific utility function or the absolute values of the measured reward.…”

Section: Construct Validitymentioning

confidence: 56%

“…However, a representative model of the nature of the source trace including FGS, IAT, and FET of the failure occurrences is missing from [50][51][52]. The FGS, IAT, and FET of the failure occurrences are cleary defined via statistical distributions in [53,54]. A probabilistic failure model not fitted to real data is used in [53] to generate multiple failure traces, and [54] employs multiple failure traces generated by probabilistic failure model fitted to real data.…”

Section: Rq2mentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Self-Healing Systems: An Analysis of the State-of-the-Art and Required Improvements

Ghahremani

Giese

2020

Computers

Self Cite

View full text Add to dashboard Cite

Evaluating the performance of self-adaptive systems is challenging due to their interactions with often highly dynamic environments. In the specific case of self-healing systems, the performance evaluations of self-healing approaches and their parameter tuning rely on the considered characteristics of failure occurrences and the resulting interactions with the self-healing actions. In this paper, we first study the state-of-the-art for evaluating the performances of self-healing systems by means of a systematic literature review. We provide a classification of different input types for such systems and analyse the limitations of each input type. A main finding is that the employed inputs are often not sophisticated regarding the considered characteristics for failure occurrences. To further study the impact of the identified limitations, we present experiments demonstrating that wrong assumptions regarding the characteristics of the failure occurrences can result in large performance prediction errors, disadvantageous design-time decisions concerning the selection of alternative self-healing approaches, and disadvantageous deployment-time decisions concerning parameter tuning. Furthermore, the experiments indicate that employing multiple alternative input characteristics can help with reducing the risk of premature disadvantageous design-time decisions.

show abstract

Training Prediction Models for Rule-Based Self-Adaptive Systems

Cited by 16 publications

References 21 publications

Improving Scalability and Reward of Utility-Driven Self-Healing for Large Dynamic Architectures

Improving Scalability and Reward of Utility-Driven Self-Healing for Large Dynamic Architectures

All Versus One: An Empirical Comparison on Retrained and Incremental Machine Learning for Modeling Performance of Adaptable Software

Evaluation of Self-Healing Systems: An Analysis of the State-of-the-Art and Required Improvements

Contact Info

Product

Resources

About