Sharing experiments using open‐source software

Nelson, Adam G.

doi:10.1002/spe.1004

Cited by 16 publications

(7 citation statements)

References 11 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, our industrycollaborated project experience shows that once the model is built by researchers, its adoption/implementation by practitioners is a pretty straightforward process [65]- [68]. This section provides an in depth discussion and alternative solutions for such practitioners.…”

Section: Learning Curvementioning

confidence: 99%

On the Value of Ensemble Effort Estimation

Kocagüneli

Menzies

Keung

2012

IIEEE Trans. Software Eng.

Self Cite

222

210

View full text Add to dashboard Cite

Abstract-Background: Despite decades of research, there is no consensus on which software effort estimation methods produce the most accurate models. Aim: Prior work has reported that, given M estimation methods, no single method consistently outperforms all others. Perhaps rather than recommending one estimation method as best, it is wiser to generate estimates from ensembles of multiple estimation methods. Method: 9 learners were combined with 10 pre-processing options to generate 9 × 10 = 90 solo-methods. These were applied to 20 data sets and evaluated using 7 error measures. This identified the best n (in our case n = 13) solo-methods that showed stable performance across multiple datasets and error measures. The top 2, 4, 8 and 13 solo-methods were then combined to generate 12 multi-methods, which were then compared to the solo-methods. Results: (i) The top 10 (out of 12) multi-methods significantly out-performed all 90 solo-methods. (ii) The error rates of the multimethods were significantly less than the solo-methods. (iii) The ranking of the best multi-method was remarkably stable. Conclusion: While there is no best single effort estimation method, there exist best combinations of such effort estimation methods.

show abstract

Section: Learning Curvementioning

confidence: 99%

On the Value of Ensemble Effort Estimation

Kocagüneli

Menzies

Keung

2012

IIEEE Trans. Software Eng.

Self Cite

222

210

View full text Add to dashboard Cite

show abstract

“…To check this if this heuristic generates inaccurate dimensions, elsewhere [13] we have conducted extensive experiments with the FASTMAP heuristic versus other, more considered clustering methods such as k-means. Our results agree with those of Faloutsos & Lin [11]: in practice, the approximate dimensions found by FASTMAP does not degrade inferencing (compared to other more complete, and slower, approaches).…”

Section: Initial Two Dimensionsmentioning

confidence: 99%

Local vs. global models for effort estimation and defect prediction

Menzies

Butcher

Marcus

et al. 2011

2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011)

Self Cite

130

149

View full text Add to dashboard Cite

Abstract-Data miners can infer rules showing how to improve either (a) the effort estimates of a project or (b) the defect predictions of a software module. Such studies often exhibit conclusion instability regarding what is the most effective action for different projects or modules.This instability can be explained by data heterogeneity. We show that effort and defect data contain many local regions with markedly different properties to the global space. In other words, what appears to be useful in a global context is often irrelevant for particular local contexts.This result raises questions about the generality of conclusions from empirical SE. At the very least, SE researchers should test if their supposedly general conclusions are valid within subsets of their data. At the very most, empirical SE should become a search for local regions with similar properties (and conclusions should be constrained to just those regions).

show abstract

“…Examples of metrics are the Chidamber and Kemerer's object-oriented (CK) metrics [10,6], structural metrics [3] or process metrics [29]. Examples of algorithms are logistic regression used by Zimmermann et al [40]; Multi-Layer Perceptron (MLP), radial basis function (RBF), k-nearest neighbor (KNN), regression tree (RT), dynamic evolving neuro-fuzzy inference system (DENFIS), and Support Vector Regression (SVR) used by Elish [14]; Bayesian networks used by Bechta [31]; and Naive Bayes, J48, Alternative Decision Tree (ADTree), and One-R considered by Nelson et al [30]. Recently, other researchers have proposed further advanced machine learning techniques, such as ensemble learning [23], clustering algorithms [36], and combined techniques [32].…”

Section: Previous Workmentioning

confidence: 99%

A Search-based Training Algorithm for Cost-aware Defect Prediction

Panichella

Alexandru

Panichella

et al. 2016

Proceedings of the Genetic and Evolutionary Computation Conference 2016

View full text Add to dashboard Cite

Research has yielded approaches to predict future defects in software artifacts based on historical information, thus assisting companies in effectively allocating limited development resources and developers in reviewing each others' code changes. Developers are unlikely to devote the same effort to inspect each software artifact predicted to contain defects, since the effort varies with the artifacts' size (cost) and the number of defects it exhibits (effectiveness). We propose to use Genetic Algorithms (GAs) for training prediction models to maximize their cost-effectiveness. We evaluate the approach on two well-known models, Regression Tree and Generalized Linear Model, and predict defects between multiple releases of six open source projects. Our results show that regression models trained by GAs significantly outperform their traditional counterparts, improving the cost-effectiveness by up to 240%. Often the top 10% of predicted lines of code contain up to twice as many defects. ABSTRACTResearch has yielded approaches to predict future defects in software artifacts based on historical information, thus assisting companies in effectively allocating limited development resources and developers in reviewing each others' code changes. Developers are unlikely to devote the same effort to inspect each software artifact predicted to contain defects, since the effort varies with the artifacts' size (cost) and the number of defects it exhibits (effectiveness). We propose to use Genetic Algorithms (GAs) for training prediction models to maximize their cost-effectiveness. We evaluate the approach on two well-known models, Regression Tree and Generalized Linear Model, and predict defects between multiple releases of six open source projects. Our results show that regression models trained by GAs significantly outperform their traditional counterparts, improving the cost-effectiveness by up to 240%. Often the top 10% of predicted lines of code contain up to twice as many defects.

show abstract

Sharing experiments using open‐source software

Cited by 16 publications

References 11 publications

On the Value of Ensemble Effort Estimation

On the Value of Ensemble Effort Estimation

Local vs. global models for effort estimation and defect prediction

A Search-based Training Algorithm for Cost-aware Defect Prediction

Contact Info

Product

Resources

About