Artificial intelligence in safety-critical systems: a systematic review

Abstract

PurposeThis study is a systematic literature review of the application of artificial intelligence (AI) in safety-critical systems. The authors aim to present the current application status according to different AI techniques and propose some research directions and insights to promote its wider application.Design/methodology/approachA total of 92 articles were selected for this review through a systematic literature review along with a thematic analysis.FindingsThe literature is divided into three themes: interpretable method, explain model behavior and reinforcement of safe learning. Among AI techniques, the most widely used are Bayesian networks (BNs) and deep neural networks. In addition, given the huge potential in this field, four future research directions were also proposed.Practical implicationsThis study is of vital interest to industry practitioners and regulators in safety-critical domain, as it provided a clear picture of the current status and pointed out that some AI techniques have great application potential. For those that are inherently appropriate for use in safety-critical systems, regulators can conduct in-depth studies to validate and encourage their use in the industry.Originality/valueThis is the first review of the application of AI in safety-critical systems in the literature. It marks the first step toward advancing AI in safety-critical domain. The paper has potential values to promote the use of the term “safety-critical” and to improve the phenomenon of literature fragmentation.

Influential factors in the adoption of clinical decision support systems in hospital settings: a systematic review and meta-synthesis of qualitative studies

“…This study conducts a systematic review and meta-synthesis of the literature on CDSS procurement. According to Wang and Chung (2022, p. 444), a “Systematic literature review is a well-established and rigorous method of evaluating and reviewing research literature based on reproducible, scientific and transparent process, which is a key tool for building an evidence base and reducing bias”. In addition to serving the complementary purposes of secondary study analysis, meta-synthesis complements quantitative SRs by investigating research questions from a different angle, bringing diversity to the development of evidence-based practice and policy (Major and Savin-Baden, 2010).…”

Maritime Anomaly Detection for Vessel Traffic Services: A Survey

“…The literature shows that the results of NN-based anomaly detection are promising. Be that as it may, the decision finding process of NN-based systems intrinsically lack explainability [12,83,84]. Due to this, the application of NN in safety-critical environments such as VTS is problematic.…”

Exploring Explainable Artificial Intelligence Techniques for Interpretable Neural Networks in Traffic Sign Recognition Systems

“…The intricacies of their internal mechanisms and decision-making processes often lack transparency, resulting in a "black-box" scenario where the logic behind specific outcomes is not clear. This ambiguity not only poses challenges in validating and correcting any biases or limitations within TSR systems, but also poses significant challenges in safety-critical applications where understanding and trusting the system's decisions are crucial [7][8][9]. It undermines the trust and validation of the system decisions that are vital in safety-focused real-world applications.…”