Four studies, two methods, one accident – An examination of the reliability and validity of Accimap and STAMP for accident analysis

Abstract

The validity and reliability of human factors and safety science methods are some of the important criteria for judging their appropriateness and utility for accident analysis, however these are rarely assessed. The aim of this study is to take a closer look at the validity and reliability of two systemic accident analysis methods (Accimap and STAMP) by comparing the results of four studies which analysed the same accident (the South Korea Sewol Ferry accident) using two methods. Studies 1 and 2 used Accimap whilst Studies 3 and 4 applied STAMP. The four studies were compared in terms of analysis procedure taken, level of detail, causal factors identified, and the recommendations for improvements suggested by the methods. The results of the causal factor comparison indicate that the reliability (degree of overlap of causal factors identified from the same method, i.e. inter-analyst overlap) of STAMP (65%) is higher than Accimap (38%). The validity (degree of overlap of causal factors identified from two different methods) is as low as 8%. The comparison of recommendations indicates that STAMP-based analyses produce a wider range of recommendations across multiple system levels while Accimap-based analyses tend to focus on whole system-related recommendations. These findings suggest that the use of a more structured method like STAMP can help produce a more reliable accident analysis results.

Intuition, the Accimap, and the question “why?” Identifying and classifying higher‐order factors contributing to road traffic collisions

“…For example, the “Five Whys” technique is described as a Root Cause Analysis method, despite being more about the investigation (if using Waterson and colleague's distinction), and the Accimap is often referred to in terms of its support for contributory factor identification (hence, investigation; see e.g., Hamim et al, 2020a; Salmon et al, 2020). The confusion between the two concepts is highlighted in the following phrase from an article authored by, among others, Waterson; “the degree to which the accident analysis method successfully identifies the causes of an accident ” (Goncalves Filho et al, 2019, our own emphasis). Although this topic likely merits greater attention than we can give it here, it is sufficient to argue the combination of Accimaps and the whys to be useful to both investigation and analysis, if such a dichotomy exists, as the benefit primarily comes in the structure it gives to creative thinking in the context of causal factor identification and consideration.…”

Exploring the Underlying Causes of Chinese Eastern Star, Korean Sewol, and Thai Phoenix Ferry Accidents by Employing the HFACS-MA

“…Finally, HFACS was the only method used in this research. While this research was practical, it might be helpful to compare different analytical methods such as 2-4 Model [47], AcciMap [48], STAMP [49], and FRAM [50].…”

An Accident Model with Considering Physical Processes for Indoor Environment Safety

“…Accimap is a generic approach and does not use taxonomies (that is different than that of the STPA [2,9]) of failures across the different levels of considered [41]. The Accimap produces less reliable accident analysis results compared to STAMP [42].…”

“…Systemic approaches related to complex sociotechnical systems have their own strengths. For one such system, it is better to adopt multiple approaches which supplement each other, even though STAMP is considered much more effective and reliable in understanding accidents and hazard analysis [10,41,42].…”