Model-based security testing: a taxonomy and systematic classification

Abstract

Model-based security testing relies on models to test whether a software system meets its security requirements. It is an active research field of high relevance for industrial applications, with many approaches and notable results published in recent years. This article provides a taxonomy for model-based security testing approaches. It comprises filter criteria (i.e. model of system security, security model of the environment and explicit test selection criteria) as well as evidence criteria (i.e. maturity of evaluated system, evidence measures and evidence level). The taxonomy is based on a comprehensive analysis of existing classification schemes for model-based testing and security testing. To demonstrate its adequacy, 119 publications on model-based security testing are systematically extracted from the five most relevant digital libraries by three researchers and classified according to the defined filter and evidence criteria. On the basis of the classified publications, the article provides an overview of the state of the art in model-based security testing and discusses promising research directions with regard to security properties, coverage criteria and the feasibility and return on investment of model-based security testing. 120 M. FELDERER ET AL. 9126 [4] defining security as a functional quality characteristic. However, it seems desirable that security testing directly targets the previous security properties, as opposed to taking the detour of functional tests of security mechanisms. This view is supported by the ISO/IEC 25010 [2] standard that revises ISO/IEC 9126 and introduces security as a new quality characteristic that is not included in the characteristic functionality any more.Because the former kind of (non-functional) security properties describes all executions of a system, this kind of security testing is intrinsically hard. Because testing cannot show the absence of faults, an immediately useful perspective directly considers the violation of these properties. This has resulted in the development of specific testing techniques such as penetration testing that simulates attacks to exploit vulnerabilities. Penetration tests are difficult to craft because tests often do not directly cause observable security exploits, and because the testers must think like an attacker [5], which requires specific expertise. During penetration testing, testers build a mental model of security properties, security mechanisms and possible attacks against the system and its environment. It seems intuitive that security testing can benefit from specifying these security test models in an explicit and processable way. Security test models provide guidance for the systematic and effective specification and documentation of security test objectives and security test cases, as well as for their automated generation and evaluation.The variant of testing that relies on explicit models that encode information on the system under test and/or its environment is called model-based testing (MBT) [6,7]. Especially in ...

How is Security Testing Done in Agile Teams? A Cross-Case Analysis of Four Software Teams

“…Also, a general understanding of the return of investment of model-based security testing approaches, which has already been highlighted as a challenge in [17], would help to apply such approaches efficiently. The issue of efficiently applying model-based testing approaches becomes even more critical when agile teams develop systems where the connection between safety and security is essential as in modern Internet-of-Things applications.…”

Similarity testing for role-based access control systems

“…However, its current version (OASIS 2014) does not support the specification of SSoD and DSoD constraints. Moreover, since the effectiveness of test criteria is strongly related to its ability to represent specific domain faults (Felderer et al 2015), there is no guarantee that similarity testing can be as effective on RBAC as they were on XACML and LTS.…”

“…This concept has been investigated under MBT (Cartaxo et al 2011), access control testing (Bertolino et al 2015) and software product line (SPL) testing (Henard et al 2014) domains, but it has never been applied to RBAC. Moreover, since the fault detection effectiveness of test criteria are strongly related to its ability to represent faults of specific domains (Felderer et al 2015), similarity testing may not be necessarily effective on RBAC domain.…”

Knowledge-based security testing of web applications by logic programming

“…, X n } is a set of variables. 5 Further, C = E, A, AP, G, T , with disjoint sets of symbols, where E is a set of exploits, A is a set of attacks AP is a set of attack patterns, G is a set of attack goals, and T is a set of types. 6 The predicates described by set F are functions over C with…”

“…Resulting model-based security testing (MBST) approaches [5] thus immediately benefit from MBT in various aspects, e.g., a high degree of automation, potential early detection of software bugs already at the design level, and high coverage of the SUT by the resulting high quality test cases [6]. A crucial role in MBST is occupied by various kinds of security models, i.e., threat, fault and risk models, and weakness and vulnerability models.…”