An evidential reasoning approach for assessing confidence in safety evidence

Abstract

Abstract-Safety cases present the arguments and evidence that can be used to justify the acceptable safety of a system. Many secondary factors such as the tools used, the techniques applied, and the experience of the people who created the evidence, can affect an assessor's confidence in the evidence cited by a safety case. One means of reasoning about this confidence and its inherent uncertainties is to present a 'confidence argument' that explicitly justifies the provenance of the evidence used. In this paper, we propose a novel approach to automatically construct these confidence arguments by enabling assessors to provide individual judgements concerning the trustworthiness and the appropriateness of the evidence. The approach is based on Evidential Reasoning and enables the derivation of a quantified aggregate of the overall confidence. The proposed approach is supported by a prototype tool (EviCA) and has been evaluated using the Technology Acceptance Model.

Software Assurance in an Uncertain World

“…To assess uncertainties which may affect the system's safety, researchers have proposed techniques to estimate confidence in structured assurance cases, either through qualitative or quantitative approaches [27,44]. The majority of these are based on the Dempster-Shafer Theory [31,60], Josang's Opinion Triangle [17], Bayesian Belief Networks (BNNs) [16,61], Evidential Reasoning (ER) [45] and weighted averages [59]. The approaches which use BBNs treat safety goals as nodes in the network and try to compute their conditional probability based on given probabilities for the leaf nodes of the network.…”

“…The area of system dependability has produced a significant body of work describing how to model assurance cases (e.g., [4,5,14,38]), and how to assess reviewer's confidence in the argument being made (e.g., [16,31,45,59,60]). There is also early work on assessing the impact of change on the assurance argument when the system undergoes change [39].…”

Modelling confidence in railway safety case

“…The quantitative assessment framework proposed in this paper could be compared to studies like the paper of Nair et al (2015), but only for the expert judgement extraction. They also provide confidence propagation calculation rules based on belief theory.…”

An analysis of safety evidence management with the Structured Assurance Case Metamodel

“…SACM explicitly indicates that evidence evaluations provide information about the relationships of evidence items with subject claims (i.e., claims in an argument structure), and we consider that most of the content of this section should be, or already is, part of the Argumentation Metamodel. Much information about evidence evaluations seems to correspond to confidence arguments [48], which aim to justify the adequacy of a given argument structure. All argumentation-related aspects should be moved to the Argumentation Metamodel.…”

“…The Argumentation Metamodel also allows the use of Asserted Evidence and Asserted Counter Evidence for modelling evidentiary support of Information Element to more than one claim. We think that this can lead to weak or inadequate argumentation structures because some characteristics of a piece of evidence depend on single claims [48]. For example, the use of a hazard log as evidence could be appropriate for a claim such as "System hazards have been identified and recorded" but not for "All the possible system hazards have been mitigated or avoided".…”