Designing Robust Software Systems through Parametric Markov Chain Synthesis

Abstract

Abstract-We present a method for the synthesis of software system designs that satisfy strict quality requirements, are Paretooptimal with respect to a set of quality optimisation criteria, and are robust to variations in the system parameters. To this end, we model the design space of the system under development as a parametric continuous-time Markov chain (pCTMC) with discrete and continuous parameters that correspond to alternative system architectures and to the ranges of possible values for configuration parameters, respectively. Given this pCTMC and required tolerance levels for the configuration parameters, our method produces a sensitivity-aware Pareto-optimal set of designs, which allows the modeller to inspect the ranges of quality attributes induced by these tolerances, thus enabling the effective selection of robust designs. Through application to two systems from different domains, we demonstrate the ability of our method to synthesise robust designs with a wide spectrum of useful tradeoffs between quality attributes and sensitivity.Keywords-software performance and reliability engineering; probabilistic model synthesis; multi-objective optimisation I. INTRODUCTION Evaluating the performance, reliability and other quality attributes of alternative designs is essential for the cost-effective engineering of software [1], [2]. Delaying this evaluation until integration or system testing can greatly increase engineering costs, as defects identified late in the development lifecycle require much more effort to fix [3]. A common method to avoid this delay uses model-based simulation [4] or formal verification [5] to predict the quality attributes of alternative designs. Models that meet the quality requirements of the system under development are then used as a basis for its implementation. Models based on queueing networks [6], probabilistic models [2], [5] and timed automata [7] have been used for this purpose, together with tools for their simulation (e.g. Palladio [8]) and verification (e.g. PRISM [9]). Furthermore, recently proposed approaches automate the search for suitable designs. Probabilistic model repair [10], [11] automatically modifies the transition probabilities of Markov models that violate a quality requirement, generating new models that meet the requirement. Precise parameter synthesis [12] identifies transition rates that enable continuous Markov models to satisfy a quality requirement or to optimise a quality attribute of the modelled system. Finally, probabilistic model synthesis [13] starts from a design template that captures alternative system designs, and uses multiobjective optimisation to generate the Pareto-optimal set of Markov models associated with the quality requirements of the system.

How to Evaluate Solutions in Pareto-Based Search-Based Software Engineering: A Critical Review and Methodological Guidance

“…IGD [26] is a well-known indicator in the field (e.g., in [5], [13], [32], [37], [39], [43], [52], [80], [89], [91], [126], [131]). As the name suggests, IGD, an inversion of GD, is to measure how close the Pareto front is to the obtained solution set.…”

HaiQ

“…In contrast, product line reliability analysis approaches [16,17,26,41] can analyze collections of system designs encoded in feature models individually or collectively. A recent approach to continuous-time probabilistic design synthesis [12] uses a template-based solution to analyze alternative designs, but assumes an existing encoding of design options in a set of discrete variables and does not systematically enforce any structural constraints in the designs. Compared with these approaches, HaiQ's focus is not only on variability, but also on structure, being able to synthesize design alternatives that satisfy complex structural constraints.…”

Socio-Cyber-Physical Systems: Models, Opportunities, Open Challenges

“…Ideally, these techniques and tools would start from a set of SCPS requirements, and would present developers with alternative systems designs that satisfy the requirements and are Pareto-optimal with respect to multiple optimisation criteria such as cost, utility and environmental impact. Such solutions have been proposed recently for the development of software systems (e.g., [24], [25]), and can also handle uncertainty in the operational profile of the system under development [26], [27]. However, these solutions cannot yet handle some of the new types of requirements encountered in SCPS.…”