Comparing Hyperparameter Optimization in Cross- and Within-Project Defect Prediction: A Case Study

Öztürk, Muhammed Maruf

doi:10.1007/s13369-018-3564-9

Cited by 5 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we compare our results with the previous studies according to the optimization methods, classifiers and hyper-parameters used for optimization and what the results were and are thus able to discover if our study is an improvement on the existing ones or not. As in papers [2] and [12], the 30 classification algorithm parameters were tuned, in paper [3] and [34] only 2 classification algorithms parameters were tuned and in paper [13] SVM-RBF parameters were optimized using AIS (clonal selection) and so its results are also comparable with our results due to classifiers and optimization methods used. Hence the results gratifyingly showed that our technique was indeed an improvement on the existing ones.…”

Section: Comparison With Previous Techniquessupporting

confidence: 68%

“…They reported that for the German dataset the three objectives were improved and indicating that if fairness and performance of multi-objective optimization are understood, then it is possible to achieve one without affecting the other one. öztürk et al [34] investigated the effect of hyperparameter optimization in cross-project defect prediction (CPDP) and within-project defect prediction (WPDP). For this they have used RF and SVM using a grid search for searching and Gradient Boosting machine for Boosting and 20 datasets from Softlab, NASA MDP, and open-source are used.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Hyper-Parameter Optimization of Classifiers, Using an Artificial Immune Network and Its Application to Software Bug Prediction

et al. 2020

View full text Add to dashboard Cite

Software testing is an important task in software development activities, and it requires most of the resources, namely, time, cost and effort. To minimize this fatigue, software bug prediction (SBP) models are applied to improve the software quality assurance (SQA) processes by predicting buggy components. The bug prediction models use machine learning classifiers so that bugs can be predicted in software components in some software metrics. These classifiers are characterized by some configurable parameters, called hyperparameters that need to be optimized to ensure better performance. Many methods have been proposed by researchers to predict the defective components but these classifiers sometimes not perform well when default settings are used for machine learning classifiers. In this paper, software bug prediction model is proposed which uses machine learning classifiers in conjunction with the Artificial Immune Network (AIN) to improve bug prediction accuracy through its hyper-parameter optimization. For this purpose, seven machine learning classifiers, such as support vector machine Radial base function (SVM-RBF), K-nearest neighbor (KNN) (Minkowski metric), KNN (Euclidean metric), Naive Bayes (NB), Decision Tree (DT), Linear discriminate analysis (LDA), Random forest (RF) and adaptive boosting (AdaBoost), were used. The experiment was carried out on bug prediction dataset. The results showed that hyper-parameter optimization of machine learning classifiers, using AIN and its applications for software bug prediction, performed better than when classifiers with their default hyper-parameters were used. INDEX TERMS Artificial immune network (AIN), artificial immune system (AIS), hyper-parameter optimization, optimized artificial immune network (opt-aiNet), software bug prediction (SBP).

show abstract

Section: Comparison With Previous Techniquessupporting

confidence: 68%

Section: Related Workmentioning

confidence: 99%

Hyper-Parameter Optimization of Classifiers, Using an Artificial Immune Network and Its Application to Software Bug Prediction

et al. 2020

View full text Add to dashboard Cite

show abstract

“…One such method is hyperparameter optimisation for an enhanced convolutional neural network (CNN) model for WPDP [4]. Another study investigated the effects of optimising WPDP hyperparameters [5]. One effective approach for reliably predicting software system bugs is the use of logistic regression and ensemble-bagged tree-based prediction models [6].…”

Section: Introductionmentioning

confidence: 99%

Heterogeneous Cross-Project Defect Prediction Using Encoder Networks and Transfer Learning

Haque,

Ali,

McClean

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Heterogeneous cross-project defect prediction (HCPDP) aims to predict defects in new software projects using defect data from previous software projects where the source and target projects have some different metrics. Most existing methods only find linear relationships in the software defect features and datasets. Additionally, these methods use multiple defect datasets from different projects as source datasets. In this paper, we propose a novel method called heterogeneous cross-project defect prediction using encoder and transfer learning (ETL). ETL uses encoders to extract the important features from source and target datasets. Also, to minimize negative transfer during transfer learning, we used an augmented dataset that contains pseudo-labels and the source dataset. Additionally, we have used very limited data to train the model. To evaluate the performance of the ETL approach, 16 datasets from four publicly available software defect projects were used. Furthermore, we compared the proposed method with four HCPDP methods namely EGW, HDP_KS, CTKCCA and EMKCA, and one WPDP method from existing literature. The proposed method on average outperforms the baseline methods in terms of PD, PF, F1-score, G-mean and AUC.

show abstract

“…Normally, classifiers can be trained based on the dataset collected previously. The existing ML techniques have been facilitated many SDP approaches [30][31][32][33]. The classifier with the highest performance index is selected by evaluating the performance in terms of balance which reflects class imbalance.…”

mentioning

confidence: 99%

HASPO: Harmony Search-Based Parameter Optimization for Just-in-Time Software Defect Prediction in Maritime Software

et al. 2021

View full text Add to dashboard Cite

Software is playing the most important role in recent vehicle innovations, and consequently the amount of software has rapidly grown in recent decades. The safety-critical nature of ships, one sort of vehicle, makes software quality assurance (SQA) a fundamental prerequisite. Just-in-time software defect prediction (JIT-SDP) aims to conduct software defect prediction (SDP) on commit-level code changes to achieve effective SQA resource allocation. The first case study of SDP in the maritime domain reported feasible prediction performance. However, we still consider that the prediction model has room for improvement since the parameters of the model are not optimized yet. Harmony search (HS) is a widely used music-inspired meta-heuristic optimization algorithm. In this article, we demonstrated that JIT-SDP can produce better performance of prediction by applying HS-based parameter optimization with balanced fitness value. Using two real-world datasets from the maritime software project, we obtained an optimized model that meets the performance criterion beyond the baseline of a previous case study throughout various defect to non-defect class imbalance ratio of datasets. Experiments with open source software also showed better recall for all datasets despite the fact that we considered balance as a performance index. HS-based parameter optimized JIT-SDP can be applied to the maritime domain software with a high class imbalance ratio. Finally, we expect that our research can be extended to improve the performance of JIT-SDP not only in maritime domain software but also in open source software.

show abstract

Comparing Hyperparameter Optimization in Cross- and Within-Project Defect Prediction: A Case Study

Cited by 5 publications

References 34 publications

Hyper-Parameter Optimization of Classifiers, Using an Artificial Immune Network and Its Application to Software Bug Prediction

Hyper-Parameter Optimization of Classifiers, Using an Artificial Immune Network and Its Application to Software Bug Prediction

Heterogeneous Cross-Project Defect Prediction Using Encoder Networks and Transfer Learning

HASPO: Harmony Search-Based Parameter Optimization for Just-in-Time Software Defect Prediction in Maritime Software

Contact Info

Product

Resources

About