On trustworthy measurements when testing dependable systems: a discussion and experiences

Abstract

The scientific literature and the industrial practice agree since several years on the fundamental role of experimental evaluation (testing) in the assessment of the dependability attributes in critical systems. This paper analyzes the key issue of achieving repeatable measurements in such systems, which is a major milestone albeit often difficult to achieve if not to approach appropriately. To improve the repeatability of the experiments and consequently improve our confidence in the results, this paper presents the support of metrological assessment of instruments and results when testing critical systems. The paper identifies the current status of the research and existing gaps, and presents three case studies developed by the authors where measurement theory is applied to aim to trustworthy, repeatable experiments.