An Empirical Validation of Learning Schemes Using an Automated Genetic Defect Prediction Framework

Murillo-Morera, Juan; Castro-Herrera, Carlos; Arroyo, Javier; Fuentes-Fernández, Rubén

doi:10.1007/978-3-319-47955-2_19

Cited by 3 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Genetic algorithms are, in general, substantially faster than exhaustive search procedures. In previous studies (Murillo-Morera et al 2016c;2016b), we have corroborated this experimentally in the area of software prediction.…”

Section: Genetic Approachsupporting

confidence: 79%

A genetic algorithm based framework for software effort prediction

Murillo-Morera

Quesada-López

Castro-Herrera

et al. 2017

J Softw Eng Res Dev

Self Cite

View full text Add to dashboard Cite

Background: Several prediction models have been proposed in the literature using different techniques obtaining different results in different contexts. The need for accurate effort predictions for projects is one of the most critical and complex issues in the software industry. The automated selection and the combination of techniques in alternative ways could improve the overall accuracy of the prediction models. Objectives: In this study, we validate an automated genetic framework, and then conduct a sensitivity analysis across different genetic configurations. Following is the comparison of the framework with a baseline random guessing and an exhaustive framework. Lastly, we investigate the performance results of the best learning schemes. Methods: In total, six hundred learning schemes that include the combination of eight data preprocessors, five attribute selectors and fifteen modeling techniques represent our search space. The genetic framework, through the elitism technique, selects the best learning schemes automatically. The best learning scheme in this context means the combination of data preprocessing + attribute selection + learning algorithm with the highest coefficient correlation possible. The selected learning schemes are applied to eight datasets extracted from the ISBSG R12 Dataset. Results: The genetic framework performs as good as an exhaustive framework. The analysis of the standardized accuracy (SA) measure revealed that all best learning schemes selected by the genetic framework outperforms the baseline random guessing by 45-80%. The sensitivity analysis confirms the stability between different genetic configurations. Conclusions: The genetic framework is stable, performs better than a random guessing approach, and is as good as an exhaustive framework. Our results confirm previous ones in the field, simple regression techniques with transformations could perform as well as nonlinear techniques, and ensembles of learning machines techniques such as SMO, M5P or M5R could optimize effort predictions.

show abstract

Section: Genetic Approachsupporting

confidence: 79%

A genetic algorithm based framework for software effort prediction

Murillo-Morera

Quesada-López

Castro-Herrera

et al. 2017

J Softw Eng Res Dev

Self Cite

View full text Add to dashboard Cite

show abstract

“…In 2016 [31], we extended this work by conducting a study of how to select the best learning schemes automatically for a specific data set, with a main focus on machine learning algorithms according to their performance (AUC) and using a genetic approach. In this study, we used twelve data sets but without compare our study with others, for example Song's framework using different search spaces (12 and 864).…”

Section: Related Workmentioning

confidence: 99%

“…In this paper, we used more data sets (seventeen) than previous works [31] and [32]. We compared our study with others.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An Automated Defect Prediction Framework using Genetic Algorithms: A Validation of Empirical Studies

Murillo-Morera¹,

Castro-Herrera²,

Arroyo³

et al. 2016

INTARTIF

View full text Add to dashboard Cite

Today, it is common for software projects to collect measurement data through development processes. With these data, defect prediction software can try to estimate the defect proneness of a software module, with the objective of assisting and guiding software practitioners. With timely and accurate defect predictions, practitioners can focus their limited testing resources on higher risk areas. This paper reports the results of three empirical studies that uses an automated genetic defect prediction framework. This framework generates and compares different learning schemes (preprocessing + attribute selection + learning algorithms) and selects the best one using a genetic algorithm, with the objective to estimate the defect proneness of a software module. The first empirical study is a performance comparison of our framework with the most important framework of the literature. The second empirical study is a performance and runtime comparison between our framework and an exhaustive framework. The third empirical study is a sensitivity analysis. The last empirical study, is our main contribution in this paper. Performance of the software development defect prediction models (using AUC, Area Under the Curve) was validated using NASA-MDP and PROMISE data sets. Seventeen data sets from NASA-MDP (13) and PROMISE (4) projects were analyzed running a N × M -fold cross-validation. A genetic algorithm was used to select the components of the learning schemes automatically, and to assess and report the results. Our results reported similar performance between frameworks. Our framework reported better runtime than exhaustive framework. Finally, we reported the best configuration according to sensitivity analysis.

show abstract