An extended systematic literature review on provision of evidence for safety certification

Abstract

Abstract:Context: Critical systems in domains such as aviation, railway, and automotive are often subject to a formal process of safety certification. The goal of this process is to ensure that these systems will operate safely without posing undue risks to the user, the public, or the environment. Safety is typically ensured via complying with safety standards. Demonstrating compliance to these standards involves providing evidence to show that the safety criteria of the standards are met. Objective:In order to cope with the complexity of large critical systems and subsequently the plethora of evidence information required for achieving compliance, safety professionals need in-depth knowledge to assist them in classifying different types of evidence, and in structuring and assessing the evidence. This paper is a step towards developing such a body of knowledge that is derived from a largescale empirically rigorous literature review. Method:We use a Systematic Literature Review (SLR) as the basis for our work. The SLR builds on 218 peer-reviewed studies, selected through a multi-stage process, from 4,963 studies published between 1990 and 2012. Results:We develop a taxonomy that classifies the information and artefacts considered as evidence for safety. We review the existing techniques for safety evidence structuring and assessment, and further study the relevant challenges that have been the target of investigation in the academic literature. We analyse commonalities in the results among different application domains and discuss implications of the results for both research and practice. Conclusion:The paper is, to our knowledge, the largest existing study on the topic of safety evidence. The results are particularly relevant to practitioners seeking a better grasp on evidence requirements as well as to researchers in the area of system safety. As a major finding of the review, the results strongly suggest the need for more practitioner-oriented and industry-driven empirical studies in the area of safety certification.

An analysis of safety evidence management with the Structured Assurance Case Metamodel

“…Their importance is also growing as a result of their recent introduction in domains such as automotive and healthcare [4]. An example of system assurance is safety certification, which can be roughly defined as a formal assurance by a third party (e.g., a certification authority) that a system fulfils its safety requirements and thus that the system does not pose undue risks to people, property, or the environment [5]. This formal assurance is typically based on the compliance with safety standards, such as IEC 61508 for electrical, electronic, and programmable electronic systems in a wide range of industries, DO-178C for avionics, the CENELEC standards for railway (e.g., EN 50128), and ISO 26262 for the The analysis has resulted in the identification of elements that need to be clarified, might be inconsistent, might not be suitable, or could be included in SACM for safety evidence management.…”

“…Other publications have indicated the potential relationship of SACM with their proposals for safety evidence lifecycle [23], for characterising safety evidence assessment [24], and for characterising safety evidence in general [25,26]. Nair et al [5] indicate that SACM does not provide a thorough and sufficiently detailed analysis of the possible evidence types to provide for safety certification and of how to structure and assess evidence. Li et al [27] consider that SACM lacks support for the evidence collection process.…”

“…To the best of our knowledge, no publication up to now has studied in depth how safety evidence management can be addressed with all the classes and associations of SACM. Regarding prior work on safety evidence management, recent publications have presented large studies on the state of the art [5] and on the state of the practice [6,7]. It is also easy to find deliverables in research projects that have reviewed the literature and industrial practices (e.g., [26]).…”

“…For example, the inexistence of residual errors cannot be demonstrated. Therefore, and as introduced above, safety evidence can be defined as artefacts that contribute to developing confidence in the safe operation of a system and that can be used to show the fulfilment of the criteria of a safety standard [5]. For example, a hazard log cannot be used as proof of a complete absence of hazards for a system, but can help a certifier develop confidence in its safe operation.…”

“…4), Evidence Item is defined as an abstract class that represents objects that are collected as evidence. For example, a hazard log would be a document, and an evidence group might be created for compiling this log and other artefacts related to safety analysis results [5], such as documents with fault tree analysis, failure mode and effect analysis, etc. In our opinion, and in line with the explanation in Section 3.1, some classes that specialise Evidence Item should not be regarded as safety evidence.…”