Local versus Global Lessons for Defect Prediction and Effort Estimation

Menzies, Tim; Butcher, Andrew; Cok, David R.; Marcus, Andrian; Layman, Lucas; Shull, Forrest; Turhan, Burak; Zimmermann, Thomas

doi:10.1109/tse.2012.83

Cited by 225 publications

(156 citation statements)

References 55 publications

Supporting

Mentioning

149

Contrasting

Unclassified

Order By: Relevance

“…More recent work has been emphasising the relatively good predictive performance achieved by ensembles of learning machines (Kultur et al 2009;Minku and Yao 2013a;Kocaguneli et al 2012) and local methods that make estimations based on completed projects similar to the project being estimated (Minku and Yao 2013a;Menzies et al 2013;Bettenburg et al 2012). For instance, Regression Trees (RTs), Bagging ensembles of MultiLayer Perceptrons (Bag + MLPs) and Bagging ensembles of RTs (Bag + RTs) have been shown to perform well across several datasets (Minku and Yao 2013a).…”

Section: For See Assuming No Chronologymentioning

confidence: 99%

“…An insightful work (Menzies et al 2013) in the context of predicting software effort and defect proneness is based on clustering WC + CC examples, and then creating prediction rules for each cluster. These prediction rules are aimed at finding features that lead to less effort or fewer defects.…”

Section: For See Assuming No Chronologymentioning

confidence: 99%

“…In an attempt to overcome these problems, the use of CC models for SEE has been investigated (Kitchenham et al 2007;Kocaguneli et al 2010;Menzies et al 2013). CC models are typically defined as those built using datasets containing data from several companies.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Which models of the past are relevant to the present? A software effort estimation approach to exploiting useful past models

Minku

Yao

2016

Autom Softw Eng

View full text Add to dashboard Cite

Software Effort Estimation (SEE) models can be used for decision-support by software managers to determine the effort required to develop a software project. They are created based on data describing projects completed in the past. Such data could include past projects from within the company that we are interested in (WC projects) and/or from other companies (cross-company, i.e., CC projects). In particular, the use of CC data has been investigated in an attempt to overcome limitations caused by the typically small size of WC datasets. However, software companies operate in non-stationary environments, where changes may affect the typical effort required to develop software projects. Our previous work showed that both WC and CC models of the past can become more or less useful over time, i.e., they can sometimes be helpful and sometimes misleading. So, how can we know if and when a model created based on past data represents well the current projects being estimated? We propose an approach called Dynamic Cross-company Learning (DCL) to dynamically identify which WC or CC past models are most useful for making predictions to a given company at the present. DCL automatically emphasizes the predictions given by these models in order to improve predictive performance. Our experiments comparing DCL against existing WC and CC approaches show that DCL is successful in improving SEE by A preliminary version of this work appeared as Minku and Yao (2012a

show abstract

Section: For See Assuming No Chronologymentioning

confidence: 99%

Section: For See Assuming No Chronologymentioning

confidence: 99%

See 1 more Smart Citation

Which models of the past are relevant to the present? A software effort estimation approach to exploiting useful past models

Minku

Yao

2016

Autom Softw Eng

View full text Add to dashboard Cite

show abstract

“…Therefore, many studies have been attempting to tackle such challenges in order to improve the predictive performance of SEE models and facilitate their use. In particular, several studies have investigated the use of Cross-Company (CC) projects to reduce the cost of collecting WC projects for training SEE models [17,26,36].…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, several studies proposed ways to tackle heterogeneity in CC SEE [20,22,26,35,36]. In particular, the approach Dycom [35] managed to drastically reduce the number of WC projects used for training while maintaining or slightly improving predictive performance in comparison with WC models.…”

Section: Introductionmentioning

confidence: 99%

Clustering Dycom

Minku

Hou

2017

Proceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering

View full text Add to dashboard Cite

Background: Software Effort Estimation (SEE) can be formulated as an online learning problem, where new projects are completed over time and may become available for training. In this scenario, a Cross-Company (CC) SEE approach called Dycom can drastically reduce the number of Within-Company (WC) projects needed for training, saving the high cost of collecting such training projects. However, Dycom relies on splitting CC projects into different subsets in order to create its CC models. Such splitting can have a significant impact on Dycom's predictive performance. Aims: This paper investigates whether clustering methods can be used to help finding good CC splits for Dycom. Method: Dycom is extended to use clustering methods for creating the CC subsets. Three different clustering methods are investigated, namely Hierarchical Clustering, K-Means, and ExpectationMaximisation. Clustering Dycom is compared against the original Dycom with CC subsets of different sizes, based on four SEE databases. A baseline WC model is also included in the analysis. Results: Clustering Dycom with K-Means can potentially help to split the CC projects, managing to achieve similar or better predictive performance than Dycom. However, K-Means still requires the number of CC subsets to be pre-defined, and a poor choice can negatively affect predictive performance. EM enables Dycom to automatically set the number of CC subsets while still maintaining or improving predictive performance with respect to the baseline WC model. Clustering Dycom with Hierarchical Clustering did not offer significant advantage in terms of predictive performance. Conclusion: Clustering methods can be an effective way to automatically generate Dycom's CC subsets.

show abstract

Testing the theory of relative dependency from an evolutionary perspective: higher dependencies concentration in smaller modules over the lifetime of software products

Bian

Parande

Koru³

et al. 2016

J Software Evolu Process

View full text Add to dashboard Cite

Recent studies conducted on the single releases of multiple software products showed that dependencies concentrate on smaller modules, that is, smaller modules have more dependencies per source line of code. This phenomenon, called the Theory of Relative Dependency, explains why some earlier studies reported that smaller modules were proportionally more defect prone. It is important to test the Theory of Relative Dependency from multiple perspectives so that it can be used as an explanatory argument when garnering organizational support to give a higher quality assurance (QA) priority to smaller modules. In this study, we test the validity of this theory from an evolutionary perspective by examining the consecutive releases of a number of software products. Dependencies do concentrate over smaller modules regardless of the product age. Furthermore, continuous refactoring efforts are associated with increasing concentration of dependencies on smaller modules over product lifetime. Based on the consistent results, software managers and developers should consider giving a higher QA priority to smaller modules over the lifetime of a software product. In the projects where refactoring is adopted continuously, the QA priority on smaller modules should be further increased as the software product ages.

show abstract

Local versus Global Lessons for Defect Prediction and Effort Estimation

Cited by 225 publications

References 55 publications

Which models of the past are relevant to the present? A software effort estimation approach to exploiting useful past models

Which models of the past are relevant to the present? A software effort estimation approach to exploiting useful past models

Clustering Dycom

Testing the theory of relative dependency from an evolutionary perspective: higher dependencies concentration in smaller modules over the lifetime of software products

Contact Info

Product

Resources

About