We propose Guided Random Testing (GRT), which uses static and dynamic analysis to include information on program types, data, and dependencies in various stages of automated test generation. Static analysis extracts knowledge from the system under test. Test coverage is further improved through state fuzzing and continuous coverage analysis. We evaluated GRT on 32 real-world projects and found that GRT outperforms major peer techniques in terms of code coverage (by 13 %) and mutation score (by 9 %). On the four studied benchmarks of Defects4J, which contain 224 real faults, GRT also shows better fault detection capability than peer techniques, finding 147 faults (66 %). Furthermore, in an in-depth evaluation on the latest versions of ten popular real-world projects, GRT successfully detects over 20 unknown defects that were confirmed by developers.

Abstract:Testing is one of the most resource-intensive activities in software development and consumes between 30 and 50% of total development costs according to many studies. Testing is however often not organized to maximize business value and not aligned with a project's mission. Path, branch, instruction, mutation, scenario, or requirement testing usually treat all aspects of software as equally important, while in practice 80% of the value often comes from 20% of the software. In order to maximize the return of investment gained from software testing, the management of testing needs to maximize its value contribution. In this chapter we motivate the need for value-based testing, describe practices supporting the management of value-based testing, outline a framework for value-based test management, and illustrate the framework with an example.

No abstract

Software Product Lines (SPLs) are families of related software systems distinguished by the set of features each one provides. Over the past decades SPLs have been the subject of extensive research and application both in academia and industry. SPLs practices have proven benefits such as better product customization and reduced time to market. Testing SPLs pose additional challenges stemming from the typically large number of product variants which make it infeasible to test every single one of them. In recent years, there has been an extensive research on applying Combinatorial Interaction Testing (CIT) for SPL testing. In this paper we present the first systematic mapping study on this subject. Our research questions aim to gather information regarding the techniques that have been applied, the nature of the case studies used for their evaluation, and what phases of CIT have been addressed. Our goal is to identify common trends, gaps, and opportunities for further research and application.

No abstract

No abstract

Risk orientation in testing is an important means to balance quality, time-tomarket, and cost of software. Especially for small and medium enterprises (SME) under high competitive and economic pressure, risk orientation can help to focus testing activities on critical areas of a software product. Although several risk-based approaches to testing are available, the topic has so far not been investigated in the context of SME, where risks are often associated with business critical issues. This article fills the gap and explores the state of risk orientation in the testing processes of SME. Furthermore, it compares the state of risk-based testing in SME to the situation in large enterprises. The article is based on a multiple case study conducted with five SME. A previous study on risk-based testing in large enterprises is used as reference for investigating the differences between risk orientation in SME and large enterprises. The findings of our study show that a strong business focus, the use of informal risk concepts, as well as the application of risk knowledge to reduce testing cost and time are key differences of risk-based testing in SME compared to large enterprises.

No abstract