Selection of regression system tests for security policy evolution

Abstract

As security requirements of software often change, developers may modify security policies such as access control policies (policies in short) according to evolving requirements. To increase confidence that the modification of policies is correct, developers conduct regression testing. However, rerunning all of existing system test cases could be costly and time-consuming. To address this issue, we develop a regression-test-selection approach, which selects every system test case that may reveal regression faults caused by policy changes. Our evaluation results show that our test-selection approach reduces a substantial number of system test cases efficiently.

Coverage-Based Test Cases Selection for XACML Policies

“…The policy set target (line 3) is empty, which means that it applies to any subject, resource, action and environment. The policy target (lines [5][6][7][8][9][10][11][12] says that this policy applies to any subject, any action, any environment and the "books" resource. This policy has a first rule (ruleA) (lines 13-34) with a target (lines [14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33] specifying that this rule applies only to the access requests of a "read" action of "books", and "documents" resources with any environment.…”

“…One of the objectives of this paper is to improve the trustworthiness of the experimental analysis presented in previous related works (like [6] and [7]) by using a larger and more representative set of XACML policies. Thus, we include in the experiment six real world policies, which differ from each other in terms of the complexity of their structure and the number of elements they include.…”

“…The work in [18] presents a survey of selection techniques able to identify the test cases that are relevant to some set of changes and addresses the emerging trends in the field. The work of [7] addresses three regression test selection techniques for security policies, based on mutation analysis, coverage analysis and recorded request evaluation respectively. They can be applied to XACML based systems and can reveal regression faults caused by policy changes, thus reducing the number of test cases.…”

“…In particular, Martin et al [6] proposed the definition of some policy coverage metrics and a greedy algorithm for test case selection able to increase the overall coverage measure. Hwang et al [7] focused on regression testing and used some monitoring facilities, directly integrated on the program code, to establish correlation between executed test cases and the entities covered in an XACML policy. XACML smart coverage differs from these works in several aspects: it integrates and extends the coverage criteria proposed in [6] by including additional constraints and XACML elements; it is not specifically conceived for regression testing, therefore it does not rely on some coverage information obtained from a previous execution of test cases; it does not require any instrumentation of program code nor any monitoring facility for coverage measurement, because it exploits only the XACML policy information and the test case specification.…”

“…XACML smart coverage differs from these works in several aspects: it integrates and extends the coverage criteria proposed in [6] by including additional constraints and XACML elements; it is not specifically conceived for regression testing, therefore it does not rely on some coverage information obtained from a previous execution of test cases; it does not require any instrumentation of program code nor any monitoring facility for coverage measurement, because it exploits only the XACML policy information and the test case specification. In addition, we improve on the trustworthiness of the experimental analyses presented in [6] and [7] by: using a set of XACML policies, that is larger and includes policies of varying structural complexity; adopting a test generation strategy that has been proven to be more effective than existing ones; considerably increasing the set of mutation operators for the evaluation of test suite effectiveness.…”