A security metric based on security arguments

Abstract

Software security metrics that facilitate decision making at the enterprise design and operations levels are a topic of active research and debate. These metrics are desirable to support deployment decisions, upgrade decisions, and so on; however, no single metric or set of metrics is known to provide universally effective and appropriate measurements. Instead, engineers must choose, for each software system, what to measure, how and how much to measure, and must be able to justify the rationale for how these measurements are mapped to stakeholder security goals. An assurance argument for security (i.e., a security argument) provides comprehensive documentation of all evidence and rationales for justifying belief in a security claim about a software system. In this work, we motivate the need for security arguments to facilitate meaningful and comprehensive security metrics, and present a novel framework for assessing security arguments to generate and interpret security metrics.

A Rational Foundation for Software Metrology

“…The model that suggests itself in this context is not measurement of an observable property, but rather a quantification of defensible belief using Bayesian, Dempster-Shafer [102], or other similar theories. The relationship between this kind of metric and an assurance case, which also focuses on evidential reasoning, was described in [103].…”

Security assurance cases—state of the art of an emerging approach

“…We also looked into the authors of the selected papers to find the portion of the papers with at least one author from industry. We found that less than 25% (12 papers) (Cockram and Lautieri 2007;Goodger et al 2012;Netkachova et al 2015;Netkachova and Bloomfield 2016;Xu et al 2017;Gacek et al 2014;Rodes et al 2014;Bloomfield et al 2017;Netkachova et al 2014;Gallo and Dahab 2015;Cheah et al 2018;Ionita et al 2017) included at least one author from industry.…”

“…We also looked into the authors of the selected papers to find the portion of the papers with at least one author from industry. We found that less than 25% (12 papers) [16,28,51,50,79,24,57,11,49,26,14,38] included at least one author from industry.…”

“…The data sources vary among the validations, as can be seen in Table 14. We categorize these sources into three main categories: -Research, open source, and in-house projects (20) (Ankrum and Kromholz 2005;Chindamaikul et al 2014;Cockram and Lautieri 2007;Coffey et al 2014;Gacek et al 2014;Haley et al 2005;Hawkins et al 2015;Mohammadi et al 2018;Netkachova et al 2015;Patu and Yamamoto 2013a;Poreddy and Corns 2011;Ray and Cleaveland 2015;Rodes et al 2014;Shortt and Weber 2015;Sklyar and Kharchenko 2019;Sljivo and Gallina 2016;Strielkina et al 2018;Tippenhauer et al 2014;Vivas et al 2011;Gallo and Dahab 2015) -Commercial products / systems (9) (Ben Othmane and Ali 2016;Ben Othmane et al 2014;Calinescu et al 2017;Cheah et al 2018;Goodger et al 2012;Górski et al 2012;Masumoto et al 2013;Xu et al 2017;Netkachova et al 2014) -Standards, regulation, and technical reports (7) (Bloomfield et al 2017;Cyra and Gorski 2007;Finnegan and McCaffery 2014b;Fung et al 2018;Graydon and Kelly 2013;He and Johnson 2012;Sklyar and Kharchenko 2017b) SACs were presented in 31 out of the 36 validations. Representing a complete SAC is mostly not possible even in small illustrative cases due to the amount of information required ...…”

“…For that there needs to be a mechanism to actively assess the quality of the arguments to gain confidence in them. This is not addressed in literature apart from a few studies, e.g., Chindamaikul et al (2014) and Rodes et al (2014). Similarly, the evidence part also needs to be assessed.…”