Towards Measuring Test Coverage of Attack Simulations

Abstract

Designing secure and reliable systems is a difficult task. Threat modeling is a process that supports the secure design of systems by easing the understanding of the system's complexity, as well as identifying and modeling potential threats. These threat models can serve as input for attack simulations, which are used to analyze the behavior of attackers within the system. To ensure the correct functionality of these attack simulations, automated tests are designed that check if an attacker can reach a certain point in the threat model. Currently, there is no way for developers to estimate the degree to which their tests cover the attack simulations and, thus, they cannot the determine the quality of their tests. To resolve this shortcoming, we analyze structural testing methods from the software engineering domain and transfer them to the threat modeling domain by following an Action Design Research approach. Further, we develop a first prototype, which is able to assess the test coverage in an automated way. This will enable threat modeler to determine the quality of their tests and, simultaneously, increase the quality of the threat models.