Multi-Objective Software Effort Estimation: A Replication Study

Tawosi, Vali; Sarro, Federica; Petrozziello, Alessio; Harman, Mark

doi:10.1109/tse.2021.3083360

Cited by 23 publications

(8 citation statements)

References 101 publications

(180 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These include the use of other diversity measures, and performance measures such as the classification stability [67] and the effort required for source code inspection [33,44]. • Investigating MOEAs other than NSGA-II (e.g., MOEA/D, IBEA, MOCell, SPEA2) in order to assess to what extent the effectiveness of MEG varies depending on the underlying multi-objective algorithm [69]. • Investigating Deep-Learning (DL) in combination with MEG, to assess if this would further increase the ensemble prediction performance.…”

Section: Discussionmentioning

confidence: 99%

MEG: Multi-objective Ensemble Generation for Software Defect Prediction

Moussa

Guizzo

Sarro

2022

Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement

Self Cite

View full text Add to dashboard Cite

Background: Defect Prediction research aims at assisting software engineers in the early identification of software defect during the development process. A variety of automated approaches, ranging from traditional classification models to more sophisticated learning approaches, have been explored to this end. Among these, recent studies have proposed the use of ensemble prediction models (i.e., aggregation of multiple base classifiers) to build more robust defect prediction models. Aims: In this paper, we introduce a novel approach based on multi-objective evolutionary search to automatically generate defect prediction ensembles. Our proposal is not only novel with respect to the more general area of evolutionary generation of ensembles, but it also advances the state-of-the-art in the use of ensemble in defect prediction. Method: We assess the effectiveness of our approach, dubbed as Multi-objective Ensemble Generation (MEG), by empirically benchmarking it with respect to the most related proposals we found in the literature on defect prediction ensembles and on multi-objective evolutionary ensembles (which, to the best of our knowledge, had never been previously applied to tackle defect prediction). Result: Our results show that MEG is able to generate ensembles which produce similar or more accurate predictions than those achieved by all the other approaches considered in 73% of the cases (with favourable large effect sizes in 80% of them). Conclusions: MEG is not only able to generate ensembles that yield more accurate defect predictions with respect to the benchmarks considered, but it also does it automatically, thus relieving the engineers from the burden of manual design and experimentation. CCS CONCEPTS• Software and its engineering → Search-based software engineering; Software defect analysis; • Computing methodologies → Ensemble methods.

show abstract

Section: Discussionmentioning

confidence: 99%

MEG: Multi-objective Ensemble Generation for Software Defect Prediction

Moussa

Guizzo

Sarro

2022

Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement

Self Cite

View full text Add to dashboard Cite

show abstract

“…In particular, DeLag uses NSGA-II [30] to build (successively-improved) Pareto-optimal solutions, while seeking new non-dominating pattern sets. We rely on NSGA-II because it has been shown to be effective on a wide variety of multi-objective search problems both within and outside the software engineering domain [26], [31], [32], [33], [34], [35], [36]. Using different search algorithms might potentially improve DeLag effectiveness, however, given the extensiveness of our experimental evaluation, we deem this investigation out of the scope of this work, and we leave it for future studies.…”

Section: Genetic Algorithmmentioning

confidence: 99%

DeLag: Using Multi-Objective Optimization to Enhance the Detection of Latency Degradation Patterns in Service-Based Systems

Traini

Cortellessa

2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Performance debugging in production is a fundamental activity in modern service-based systems. The diagnosis of performance issues is often time-consuming, since it requires thorough inspection of large volumes of traces and performance indices. In this paper we present DeLag, a novel automated search-based approach for diagnosing performance issues in service-based systems. DeLag identifies subsets of requests that show, in the combination of their Remote Procedure Call execution times, symptoms of potentially relevant performance issues. We call such symptoms Latency Degradation Patterns. DeLag simultaneously searches for multiple latency degradation patterns while optimizing precision, recall and latency dissimilarity. Experimentation on 700 datasets of requests generated from two microservice-based systems shows that our approach provides better and more stable effectiveness than three state-of-the-art approaches and general purpose machine learning clustering algorithms. DeLag is more effective than all baseline techniques in at least one case study (with p ≤ 0.05 and non-negligible effect size). Moreover, DeLag outperforms in terms of efficiency the second and the third most effective baseline techniques on the largest datasets used in our evaluation (up to 22%).

show abstract

“…Similar to previous studies on software effort estimation, we use measurements that are built upon the error (or absolute error) between the predicted value and the actual value. These measures (defined in Equations 1, 2 and 3) have been found in previous work to be unbiased towards under-or overestimations [6], [14], [30]- [32]. These measure are the Mean Absolute Error (MAE), the Median Absolute Error (MdAE), and the Standard Accuracy (SA).…”

Section: Evaluation Measuresmentioning

confidence: 99%

“…Respectively, Â12 higher than 0.5 means that the first algorithm is more likely to produce better predictions. The effect size is considered small for 0.6 ≤ Â12 < 0.7, medium for 0.7 ≤ Â12 ≤ 0.8, and large for Â12 ≥ 0.8, although these thresholds are not definitive [6].…”

Section: E Statistical Analysismentioning

confidence: 99%

“…To this end, teams mainly rely on expert estimation methods like Planning Poker and Delphi [3]. However, expert judgment has been shown to be prone to bias due to its reliance on subjective assessment [4]- [6]. This motivated several research endeavours to find automated ways to predict story points of a task given its features with the aim of avoiding inaccurate estimations by human judgement, in addition to, and more importantly to an agile team, producing consistent estimations throughout the project's lifecycle.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Investigating the Effectiveness of Clustering for Story Point Estimation

Tawosi

Al-Subaihin

Sarro

2022

2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)

Self Cite

View full text Add to dashboard Cite

Automated techniques to estimate Story Points (SP) for user stories in agile software development came to the fore a decade ago. Yet, the state-of-the-art estimation techniques' accuracy has room for improvement.In this paper, we present a new approach for SP estimation, based on analysing textual features of software issues by employing latent Dirichlet allocation (LDA) and clustering. We first use LDA to represent issue reports in a new space of generated topics. We then use hierarchical clustering to agglomerate issues into clusters based on their topic similarities. Next, we build estimation models using the issues in each cluster. Then, we find the closest cluster to the new coming issue and use the model from that cluster to estimate the SP.Our approach is evaluated on a dataset of 26 open source projects with a total of 31,960 issues and compared against both baselines and state-of-the-art SP estimation techniques.The results show that the estimation performance of our proposed approach is as good as the state-of-the-art. However, none of these approaches is statistically significantly better than more naive estimators in all cases, which does not justify their additional complexity. We therefore encourage future work to develop alternative strategies for story points estimation.The experimental data and scripts we used in this work are publicly available to allow for replication and extension.

show abstract

Multi-Objective Software Effort Estimation: A Replication Study

Cited by 23 publications

References 101 publications

MEG: Multi-objective Ensemble Generation for Software Defect Prediction

MEG: Multi-objective Ensemble Generation for Software Defect Prediction

DeLag: Using Multi-Objective Optimization to Enhance the Detection of Latency Degradation Patterns in Service-Based Systems

Investigating the Effectiveness of Clustering for Story Point Estimation

Contact Info

Product

Resources

About