Evaluation of Self-Healing Systems: An Analysis of the State-of-the-Art and Required Improvements

Ghahremani, Sona; Giese, Holger

doi:10.3390/computers9010016

Cited by 11 publications

(8 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this way, the effect of the faults on the data aggregation and how this effect can be controlled via a self-healing tuning (choice of threshold) can be isolated and studied systematically. [53], [54], [27], while the evaluated parameter space with extreme fault scales stretches the experimental evaluations. Table IV summarizes the applicability of the three fault profiles to the modeled fault scenarios.…”

Section: Experimental Methodologymentioning

confidence: 99%

“…Despite the large body of work on fault correction and fault tolerance, a recent comprehensive review of such approaches for multi-agent systems identifies as imperative the need for generalized and standardized evaluation of faulttolerance approaches [10]. Another recent systematic evaluation of 36 state-of-the-art self-healing systems from the research communities of autonomic computing, self-adaptive, self-organizing and self-managing systems (ICAC, SASO, TAAS, SEAMS) is illustrated [27]. Empirical assessments conclude that multiple input traces covering a vast spectrum of failure characteristics are required to predict the performance of a self-healing system.…”

Section: Positioning and Comparison To Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Self-Healing Dilemmas in Distributed Systems: Fault Correction vs. Fault Tolerance

Nikolic

Jubatyrov

Pournaras

2021

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

Large-scale decentralized systems of autonomous agents interacting via asynchronous communication often experience the following self-healing dilemma: fault detection inherits network uncertainties making a remote faulty process indistinguishable from a slow process. In the case of a slow process without fault, fault correction is undesirable as it can trigger new faults that could be prevented with fault tolerance that is a more proactive system maintenance. But in the case of an actual faulty process, fault tolerance alone without eventually correcting persistent faults can make systems underperforming. Measuring, understanding and resolving such self-healing dilemmas is a timely challenge and critical requirement given the rise of distributed ledgers, edge computing, the Internet of Things in several energy, transport and health applications. This paper contributes a novel and general-purpose modeling of fault scenarios during system runtime. They are used to accurately measure and predict inconsistencies generated by the undesirable outcomes of fault correction and fault tolerance as the means to improve selfhealing of large-scale decentralized systems at the design phase. A rigorous experimental methodology is designed that evaluates 696 experimental settings of different fault scales, fault profiles and fault detection thresholds in a prototyped decentralized network of 3000 nodes. Almost 9 million measurements of inconsistencies were collected in a network, where each node monitors the health status of another node, while both can defect. The prediction performance of the modeled fault scenarios is validated in a challenging application scenario of decentralized and dynamic in-network data aggregation using real-world data from a Smart Grid pilot project. Findings confirm the origin of inconsistencies at design phase and provide new insights how to tune self-healing at an early stage. Strikingly, the aggregation accuracy is well predicted as shown by high correlations and low root mean square errors.

show abstract

Section: Experimental Methodologymentioning

confidence: 99%

Section: Positioning and Comparison To Related Workmentioning

confidence: 99%

Self-Healing Dilemmas in Distributed Systems: Fault Correction vs. Fault Tolerance

Nikolic

Jubatyrov

Pournaras

2021

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

show abstract

“…All values of the feature vector are in the range [0, 1]. Regression relies on the ordinary least squares model 10 and its Python implementation of the statsmodels 11 . The prediction based on linear regression is validated with two schemes.…”

Section: Experimental Methodologymentioning

confidence: 99%

“…Despite the large body of work on fault-correction and fault-tolerance, a recent comprehensive review of such approaches for multi-agent systems identifies as imperative the need for generalized and standardized evaluation of faulttolerance approaches [10]. Another recent systematic evaluation of 36 state-of-the-art self-healing systems from the research communities of autonomic computing, self-adaptive, self-organizing and self-managing systems (ICAC, SASO, TAAS, SEAMS) is illustrated [11]. Empirical assessments conclude that multiple input traces covering a vast spectrum of failure characteristics are required to predict the performance of a self-healing system.…”

Section: Positioning and Comparison To Related Workmentioning

confidence: 99%

Self-healing Dilemmas in Distributed Systems: Fault Correction vs. Fault Tolerance

Nikolić¹,

Jubatyrov²,

Pournaras³

2020

Preprint

View full text Add to dashboard Cite

Large-scale decentralized systems of autonomous agents interacting via asynchronous communication often experience the following self-healing dilemma: Fault-detection inherits network uncertainties making a faulty process indistinguishable from a slow process. The implications can be dramatic: Self-healing mechanisms become biased and cost-ineffective. In particular, triggering an undesirable fault-correction results in new faults that could be prevented with fault-tolerance instead. Nevertheless, fault-tolerance alone without eventually correcting persistent faults makes systems underperforming as well. Measuring, understanding and resolving such self-healing dilemmas is a timely challenge and critical requirement given the rise of distributed ledgers, edge computing, the Internet of Things in several application domains of energy, transport and health. This paper introduces a novel and general-purpose modeling of fault scenarios. They can accurately measure and predict inconsistencies generated by fault-correction and fault-tolerance when each node in a network can monitor the health status of another node, while both can defect. In contrast to related work, no information about the computational/application scenario, overlying algorithms or application data is required. A rigorous experimental methodology is designed that evaluates 696 experimental settings of different fault scales, fault profiles and fault detection thresholds, each with almost 9 million measurements of inconsistencies in a prototyped decentralized network of 3000 nodes. The prediction performance of the modeled fault scenarios is validated in a challenging application scenario of decentralized and dynamic in-network aggregation using real-world data from a Smart Grid pilot project. Findings confirm the origin of inconsistencies at design phase and provide new insights how to tune self-healing mechanisms at design phase. Strikingly, the aggregation accuracy is well predicted as shown by high correlations and low root mean square errors when calibration methods with application-independent features are applied.

show abstract

“…In their paper Evaluation of Self-Healing Systems: An Analysis of the State of the Art and Required Improvements [22], the authors Ghahremani and Giese study the state of the art for evaluating the performance of self-healing systems by means of a systematic literature review. They provide a classification of different input types for such systems and analyze the limitations of each input type.…”

Section: Contributions To This Special Issuementioning

confidence: 99%

Introduction to the Special Issue “Applications in Self-Aware Computing Systems and their Evaluation”

Krupitzer

Eberhardinger²,

Gerostathopoulos

et al. 2020

Computers

View full text Add to dashboard Cite

The joint 1st Workshop on Evaluations and Measurements in Self-Aware Computing Systems (EMSAC 2019) and Workshop on Self-Aware Computing (SeAC) was held as part of the FAS* conference alliance in conjunction with the 16th IEEE International Conference on Autonomic Computing (ICAC) and the 13th IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO) in Umeå, Sweden on 20 June 2019. The goal of this one-day workshop was to bring together researchers and practitioners from academic environments and from the industry to share their solutions, ideas, visions, and doubts in self-aware computing systems in general and in the evaluation and measurements of such systems in particular. The workshop aimed to enable discussions, partnerships, and collaborations among the participants. This special issue follows the theme of the workshop. It contains extended versions of workshop presentations as well as additional contributions.

show abstract

Evaluation of Self-Healing Systems: An Analysis of the State-of-the-Art and Required Improvements

Cited by 11 publications

References 71 publications

Self-Healing Dilemmas in Distributed Systems: Fault Correction vs. Fault Tolerance

Self-Healing Dilemmas in Distributed Systems: Fault Correction vs. Fault Tolerance

Self-healing Dilemmas in Distributed Systems: Fault Correction vs. Fault Tolerance

Introduction to the Special Issue “Applications in Self-Aware Computing Systems and their Evaluation”

Contact Info

Product

Resources

About