An objective comparison of the cost effectiveness of three testing methods

Yates, Derek; Malevris, Nicos

doi:10.1016/j.infsof.2006.10.009

Cited by 5 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, guidelines for the application of mutation testing are also given, by measuring the ability of killing mutants utilising test paths. The reason behind the choice of candidate sets of paths as a measurement of the effort stems from the observation that infeasible paths greatly influence the effort associated with the testing process [6,7]. This is evident from the following experiments where a huge number of paths are analysed in order to find a small number of feasible paths, see Section 6.3 for a discussion on the influence of infeasible paths on the testing effort.…”

Section: The Methods Usedmentioning

confidence: 99%

“…Following the suggestions made by Offutt and Untch [5] and Yates and Malevris [7] the application of a coverage criterion must be cost-effective. ''Software testing is an imperfect science and there is no reason for coverage to be exact'' as advocated in [5].…”

Section: Effectiveness Of the Path Generation Methodsmentioning

confidence: 99%

“…The existence and influence of infeasible paths in practice, plays a major role as it compels performing steps 1 and 2 iteratively in order to construct a set of data D that covers the target code elements. This difficulty can be extremely problematic [7,14] and results in unbearable overheads as the conducted experiment shows, see Section 6.3 for a discussion of this issue.…”

Section: Testing Based On Selected Pathsmentioning

confidence: 99%

“…The one employed in the present study is based on selecting paths that the mutants lie on, and subsequently executing them with test data, thus utilising each one of the mutants in order to kill them. The drawbacks of such a strategy, as it has been reported in the literature, are attributed to the presence of infeasible paths [6,7]. The strategy adopted in this exercise, seeks to reduce the undesirable effects of infeasible paths during the testing process and hence reciprocally, also reduce the overall testing effort.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Mutation based test case generation via a path selection strategy

Papadakis

Malevris

2012

Information and Software Technology

View full text Add to dashboard Cite

a b s t r a c tContext: Generally, mutation analysis has been identified as a powerful testing method. Researchers have shown that its use as a testing criterion exercises quite thoroughly the system under test while it achieves to reveal more faults than standard structural testing criteria. Despite its potential, mutation fails to be adopted in a widespread practical use and its popularity falls significantly short when compared with other structural methods. This can be attributed to the lack of thorough studies dealing with the practical problems introduced by mutation and the assessment of the effort needed when applying it. Such an incident, masks the real cost involved preventing the development of easy and effective to use strategies to circumvent this problem. Objective: In this paper, a path selection strategy for selecting test cases able to effectively kill mutants when performing weak mutation testing is presented and analysed. Method: The testing effort is highly correlated with the number of attempts the tester makes in order to generate adequate test cases. Therefore, a significant influence on the efficiency associated with a test case generation strategy greatly depends on the number of candidate paths selected in order to achieve a predefined coverage goal. The effort can thus be related to the number of infeasible paths encountered during the test case generation process. Results: An experiment, investigating well over 55 million of program paths is conducted based on a strategy that alleviates the effects of infeasible paths. Strategy details, along with a prototype implementation are reported and analysed through the experimental results obtained by its employment to a set of program units. Conclusion:The results obtained suggest that the strategy used can play an important role in making the mutation testing method more appealing and practical.

show abstract

Section: The Methods Usedmentioning

confidence: 99%

Section: Effectiveness Of the Path Generation Methodsmentioning

confidence: 99%

Section: Testing Based On Selected Pathsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Mutation based test case generation via a path selection strategy

Papadakis

Malevris

2012

Information and Software Technology

View full text Add to dashboard Cite

show abstract

“…One such issue is the use of mutants as a means of effectiveness evaluation. While this is a potential problem of the conducted experiment, in practice evaluating one criterion in terms of another one is a usual practice, e.g., [28]. Since the similarity approach is independent from the employed mutants, this threat should be balanced.…”

Section: Threats To Validitymentioning

confidence: 99%

Similarity testing for access control

Bertolino

Daoudagh

Kateb

et al. 2015

Information and Software Technology

View full text Add to dashboard Cite

a b s t r a c tContext: Access control is among the most important security mechanisms, and XACML is the de facto standard for specifying, storing and deploying access control policies. Since it is critical that enforced policies are correct, policy testing must be performed in an effective way to identify potential security flaws and bugs. In practice, exhaustive testing is impossible due to budget constraints. Therefore the tests need to be prioritized so that resources are focused on their most relevant subset. Objective: This paper tackles the issue of access control test prioritization. It proposes a new approach for access control test prioritization that relies on similarity. Method: The approach has been applied to several policies and the results have been compared to random prioritization (as a baseline). To assess the different prioritization criteria, we use mutation analysis and compute the mutation scores reached by each criterion. This helps assessing the rate of fault detection. Results: The empirical results indicate that our proposed approach is effective and its rate of fault detection is higher than that of random prioritization. Conclusion: We conclude that prioritization of access control test cases can be usefully based on similarity criteria.

show abstract