Taming model uncertainty in self-adaptive systems using bayesian model averaging

Abstract

Research on uncertainty quantification and mitigation of softwareintensive systems and (self-)adaptive systems, is increasingly gaining momentum, especially with the availability of statistical inference techniques (such as Bayesian reasoning) that make it possible to mitigate uncertain (quality) attributes of the system under scrutiny often encoded in the system model in terms of model parameters. However, to the best of our knowledge, the uncertainty about the choice of a specific system model did not receive the deserved attention.This paper focuses on self-adaptive systems and investigates how to mitigate the uncertainty related to the model selection process, that is, whenever one model is chosen over plausible alternative and competing models to represent the understanding of a system and make predictions about future observations. In particular, we propose to enhance the classical feedback loop of a self-adaptive system with the ability to tame the model uncertainty using Bayesian Model Averaging. This method improves the predictions made by the analyze component as well as the plan that adopts metaheuristic optimizing search to guide the adaptation decisions. Our empirical evaluation demonstrates the cost-effectiveness of our approach using an exemplar case study in the robotics domain. CCS CONCEPTS• Software and its engineering → Software system models; Software functional properties; • Computer systems organization → Self-organizing autonomic computing.

Towards Better Trust in Human-Machine Teaming through Explainable Dependability

“…We also plan to investigate how to include into the SHA model other multi-disciplinary cognitiverelated aspects and conduct variability model analysis. We want to mitigate uncertain (quality) attributes of the system under scrutiny and of the SHA model itself, especially with the availability of statistical inference techniques, such as Bayesian reasoning [7]. We would like to provide formal guarantees also for ethics-related concerns while designing, developing, deploying, and executing HMT systems.…”

Risk-driven Online Testing and Test Case Diversity Analysis for ML-enabled Critical Systems

“…We designed and carried out an experimental campaign 4 to evaluate our approach to risk-driven test case generation and diversity analysis. We executed multiple runs of the generation process by using different search approaches.…”

“…Recent empirical studies found offline testing is less effective in uncovering safety violations compared to online testing [3]. Clearly, even if the ML components are reliable, they are embedded in complex and dynamic operational ecosystems affected by sources of uncertainty that may lead to unsafe component interactions and, therefore, to accidents [4], [5]. Online testing methods for ML-enabled systems deal with these issues by considering the ML components in a closed loop with the other parts and the surrounding environment.…”

A Conceptual Framework for Explainability Requirements in Software-Intensive Systems

“…2), we say the robot is dependable. Dependability and other phenomena of interest, such as fatigue of the human agents and the outcome of the competition, are typically affected by uncertain and changing factors that shall be measured during the mission [5], [6]. Thus, both the robot and the human agents are equipped with sensors gathering information about their current state; specifically measuring their position inside the layout, the robot's level of charge, and the humans' level of muscular fatigue, which provides a measure of the physical effort they are currently enduring.…”