Safety Hazard Identification by Misuse Cases: Experimental Comparison of Text and Diagrams

Abstract

Abstract. In general, diagrams and text are both considered to have their advantages and disadvantages for the representation of use case models, but this is rarely investigated experimentally. This paper describes a controlled experiment where we compare safety hazard identification by means of misuse cases based on use case diagrams and textual use cases. The experiment participants found use case diagrams and textual use cases equally easy to use. In most cases those who used textual use cases were able to identify more failure modes or threats. The main reason for this seems to be that use cases encourage analysts to specifically focus on threats related to the functions mentioned in the use case, and textual use cases include more functional details than diagrams. The focus is decided by information in each use case which will thus decide the number of threats identified.

Graphical vs. Tabular Notations for Risk Models: On the Role of Textual Labels and Complexity

“…In the literature we found three main streams of works that compares textual and visual notations: a) studies that proposed cognitive theories to explain the differences between the notations or to explain their relative strengths (Vessey 1991), b) studies that compared different notations from a conceptual point of view (Kaczmarek et al 2015;Saleh and El-Attar 2015), and c) studies that empirically compare graphical and textual representations, e.g., for safety and system requirements (Sharafi et al 2013;Stålhane and Sindre 2008;Stålhane et al 2010;Stålhane and Sindre 2014;de la Vara et al 2016), software architectures (Heijstek et al 2011), and business processes (Ottensooser et al 2012). To the best of our knowledge, there are few similar studies that empirically investigated modeling notations for security risk (Hogganvik and Stølen 2005;Grøndahl et al 2011) or compared graphical and tabular security methods in full scale application experiments (Massacci and Paci 2012;Labunets et al 2013Labunets et al , 2014.…”

“…Regarding the studies on comprehensibility in security domain, a series of controlled experiments were conducted by Stålhane et al (Stålhane and Sindre 2008;Stålhane et al 2010;Stålhane and Sindre 2014) to compare the effectiveness of textual and graphical notations in identifying safety hazards during security requirements analysis. They compared textual use cases with system sequence diagrams (Stålhane et al 2010;Stålhane and Sindre 2014) and misuse case diagrams with textual misuse cases (Stålhane and Sindre 2008).…”

“…In short experiments, graphical methods fared better (e.g., (Hogganvik and Stølen 2005;Stålhane and Sindre 2008)). Studies with a full-fledged application for several days had either mixed results (e.g., (Massacci and Paci 2012;Labunets et al 2013Labunets et al , 2014 or concluded that the two notations were equivalent ).…”

Comparing Safety Analysis Based on Sequence Diagrams and Textual Use Cases

“…The same effect has been observed in previous experiments. We have found that FMEA outperform misuse case diagrams [17] when it comes to network problems and that misuse cases based on both diagrams and text score low on the failure modes "software unavailable", "network down" and "delete files" [18]. Thus, as a general conclusion, we will claim that TUC is better than SSD for all failure modes except those pertaining to the inner working of the computer system.…”

On the Equivalence Between Graphical and Tabular Representations for Security Risk Assessment

“…Stålhane et al have conducted a series of experiments to evaluate two representations of misuse cases: a graphical diagram and a textual template. The results reported in [23] revealed that textual use cases helped to identify more threats than use-case diagrams. In more recent experiments [24,25,26], Stålhane et al compared textual misuse cases with UML system sequence diagrams.…”