A model and sensitivity analysis of the quality economics of defect-detection techniques

Wagner, Stefan

doi:10.1145/1146238.1146247

Cited by 31 publications

(27 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Muller et al [19] present an economic model for the return on investment of test-driven development (TDD) compared to conventional development, provide a costbenefit analysis, and identify a break-even point at which TDD becomes beneficial over conventional development. As already noted, Wagner [36] proposes an analytical model of the economics of defect detection techniques that incorporates various cost factors and revenues, considering uncertainty and using sensitivity analysis to identify the factors most relevant for model simplification.…”

Section: Economic Modelsmentioning

confidence: 99%

“…Wagner [36,37] applies sensitivity analysis to his model of defect detection techniques and the COCOMO model to investigate which input factors are most important. Freimut et al [10] apply sensitivity analysis to their models of software inspection processes.…”

Section: Sensitivity Analysismentioning

confidence: 99%

“…While these models estimate costs incurred during the software development phase, other models consider issues related to postdevelopment phases. For example, Ostrand et al [34] provide a model for use in estimating the fault proneness of systems, and Wagner [36] provides a model for use in assessing the costs and benefits of defect detection techniques.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Using sensitivity analysis to create simplified economic models for regression testing

Rothermel

2008

Proceedings of the 2008 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Software engineering methodologies are subject to complex costbenefit tradeoffs. Economic models can help practitioners and researchers assess methodologies relative to these tradeoffs. Effective economic models, however, can be established only through an iterative process of refinement involving analytical and empirical methods. Sensitivity analysis provides one such method. By identifying the factors that are most important to models, sensitivity analysis can help simplify those models; it can also identify factors that must be measured with care, leading to guidelines for better test strategy definition and application. In prior work we presented the first comprehensive economic model for the regression testing process, that captures both cost and benefit factors relevant to that process while supporting evaluation of these processes across entire system lifetimes. In this work we use sensitivity analysis to examine our model analytically and assess the factors that are most important to the model. Based on the results of that analysis, we propose two new models of increasing simplicity. We assess these models empirically on data obtained by using regression testing techniques on several non-trivial software systems. Our results show that one of the simplified models assesses the relationships between techniques in the same way as the full model.

show abstract

Section: Economic Modelsmentioning

confidence: 99%

Section: Sensitivity Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using sensitivity analysis to create simplified economic models for regression testing

Rothermel

2008

Proceedings of the 2008 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…These values (cost and benefit) are symmetrical 11 . They are properly measured as either the minimum amount of money all members of society would collectively require to forego the improved infrastructure or as the maximum amount of money all members of society would collectively pay for the improved infrastructure 12 . An appropriate measure of the economic impact of an inadequate infrastructure for software testing is the profit differences of developers and users between conditions with the current testing infrastructure and conditions with the counterfactual infrastructure.…”

Section: Conceptual Economic Modelmentioning

confidence: 99%

An Impact of Linear Regression Models for Improving the Software Quality with Estimated Cost

Marandi

Khan

2015

Procedia Computer Science

View full text Add to dashboard Cite

Developing organizations are spends lots of money to finding the Errors and bugs. In this article, an application of defects removal effectiveness to improve the software quality and fault prone analysis, methods are finding the solution of parameters in linear regression models with cost estimating method. It describes the approach of quantitative quality management through defect removal effectiveness and statistical process control of cost analysis with historical project data. Software quality is going continuously monetary benefit to perform well management planning, and achieve a new height. In this methodology, Software quality model can make timely predictions of reliability indications; it's enabling to improve software development processes by target reducing the estimated cost for software products and improve the techniques for more effectively and efficiently.

show abstract

“…Muller et al [38] present an economic model for the return on investment of Test-Driven Development (TDD) compared to conventional development, provide a cost-benefit analysis, and identify a break-even point at which TDD becomes beneficial over conventional development. Wagner [57] proposes an analytical model of the economics of defect detection techniques that incorporates various cost factors and revenues, considering uncertainty and sensitivity analysis to identify the most relevant factors for model simplification. Wagner [58] also applies global sensitivity analysis (which investigates how output uncertainty can be apportioned to input factors' uncertainty) to the COCOMO model to investigate which input factors are most important.…”

Section: Prior Work On Economic Modelsmentioning

confidence: 99%

The Effects of Time Constraints on Test Case Prioritization: A Series of Controlled Experiments

Mirarab

Tahvildari

et al. 2010

IIEEE Trans. Software Eng.

139

View full text Add to dashboard Cite

Regression testing is an expensive process used to validate modified software. Test case prioritization techniques improve the cost-effectiveness of regression testing by ordering test cases such that those that are more important are run earlier in the testing process. Many prioritization techniques have been proposed and evidence shows that they can be beneficial. It has been suggested, however, that the time constraints that can be imposed on regression testing by various software development processes can strongly affect the behavior of prioritization techniques. If this is correct, a better understanding of the effects of time constraints could lead to improved prioritization techniques, and improved maintenance and testing processes. We therefore conducted a series of experiments to assess the effects of time constraints on the costs and benefits of prioritization techniques. Our first experiment manipulates time constraint levels and shows that time constraints do play a significant role in determining both the cost-effectiveness of prioritization and the relative cost-benefit tradeoffs among techniques. Our second experiment replicates the first experiment, controlling for several threats to validity including numbers of faults present, and shows that the results generalize to this wider context. Our third experiment manipulates the numbers of faults present in programs to examine the effects of faultiness levels on prioritization, and shows that faultiness level affects the relative costeffectiveness of prioritization techniques. Taken together, these results have several implications for test engineers wishing to cost-effectively regression test their software systems. These include suggestions about when and when not to prioritize, what techniques to employ, and how differences in testing processes may relate to prioritization cost-effectiveness.

show abstract

A model and sensitivity analysis of the quality economics of defect-detection techniques

Cited by 31 publications

References 23 publications

Using sensitivity analysis to create simplified economic models for regression testing

Using sensitivity analysis to create simplified economic models for regression testing

An Impact of Linear Regression Models for Improving the Software Quality with Estimated Cost

The Effects of Time Constraints on Test Case Prioritization: A Series of Controlled Experiments

Contact Info

Product

Resources

About