Predictive Models in Software Engineering: Challenges and Opportunities

Yang, Yanming; Xia, Xin; Lo, David; Bi, Tingting; Grundy, John; Yang, Xiaohu

doi:10.1145/3503509

Cited by 23 publications

(30 citation statements)

References 455 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AUC is a trade-off between the TPR and the FPR, indicating the classifier's ability to accurately predict classes. 42,48,53,54,78,87,88,90,91 10. Area under cost-effectiveness curve (AUCEC): AUCEC is a defect prediction metric that takes into account the LOC.…”

Section: Rq4: What Different Performance Measures Have Been Proposed ...mentioning

confidence: 99%

See 1 more Smart Citation

A research landscape on software defect prediction

Taskeen

Khan

Felix

2023

J Software Evolu Process

View full text Add to dashboard Cite

Software defect prediction is the process of identifying defective files and modules that need rigorous testing. In the literature, several secondary studies including systematic reviews, mapping studies, and review studies have been reported. However, no research work such as a tertiary study that combines secondary studies has focused on providing a landscape of software defect prediction useful to understand the body of knowledge. Motivated by this, we intend to perform a tertiary study by following a systematic literature review protocol to provide a research landscape of the targeted domain. We synthesize the quality of the secondary studies and investigate the employed techniques and the performance evaluation measures for evaluating the software defect prediction model. Furthermore, this study aims at exploring different datasets employed in the reported experimentation. Moreover, the current study intends at highlighting the research trends, gaps, and opportunities in the targeted research domain. The results indicate that none of the reported defect prediction techniques can be regarded as the best; however, the reported techniques performed better in different testing situations. In addition, machine learning (ML)‐based techniques perform better than traditional statistical techniques mainly due to the potential of discovering the defects and generating generalized results. Moreover, the obtained results highlight the need for further work in the domain of ML‐based techniques. Furthermore, publicly available datasets should be considered for experimentation or replication purposes. The potential future work can focus on data quality, ethical ML, cross‐project defect prediction, early defect prediction process, class imbalance problem, and model overfitting.

show abstract

Section: Rq4: What Different Performance Measures Have Been Proposed ...mentioning

confidence: 99%

“…Precision: The fraction of the total number of defective classes among the overall categorized defective class. 42,48,58,78,[90][91][92] T A B L E 1 3 Validation analysis on the basis of quality of studies in the SDP context.…”

Section: Rq4: What Different Performance Measures Have Been Proposed ...mentioning

confidence: 99%

A research landscape on software defect prediction

Taskeen

Khan

Felix

2023

J Software Evolu Process

View full text Add to dashboard Cite

show abstract

“…A related problem to ours is test suite failure prediction [PaPr21],which tries to predict the failure of the whole test suite rather than for each test case individually. A further overview of predictive methods for software engineering is given in [YXLB22].…”

Section: Related Workmentioning

confidence: 99%

OpenPredict - An Open Research Dataset and Evaluation Protocol for Fine-grained Predictive Testing

Brodmann¹,

Rodner²

2022

Angewandte Forschung in Der Wirtschaftsinformatik 2022

View full text Add to dashboard Cite

Systematic testing of every single component and interface is undoubtedly an important measure to handle the complex nature of current software systems. However, this comes with often neglected computational costs. The aim of this paper is therefore to cut time and resource needs by predictive testing, i.e., predicting test failures with machine learning using a surprisingly simple statistical feature representation. Furthermore, we present the first open research benchmark for predictive testing to enable and foster future research in this area.

show abstract

“…These metrics applied by the work of Peters et al and Shu et al include Recall (R), probability of false alarm (pf), Precision (P), F1-score (F1), and G-measure (G). The first four measures are commonly used by many studies in the mining software engineering area [26]- [28] and empirical software engineering area [15], [29]. G-measure was first introduced by Peters et al [5], while both F1-score and G-measure are harmonic means, and the G score considers the Recalls of both the majority and the minority classes [5].…”

Section: Performance Evaluationmentioning

confidence: 99%

A Deep Learning Approach to Improve Classification of Security Bug Reports

Alqahtani

2022

Preprint

View full text Add to dashboard Cite

Software developers must handle security bug reports (SBRs) before they are widely disclosed, and the system becomes vulnerable to hackers. Bug tracking systems may contain many securities-related reports which are unlabelled as SBRs. Therefore, finding unlabelled SBRs is a challenge to help security engineers identify these security issues fast and accurately. Although many methods have been proposed for classifying SBRs, challenging issues remain due to selecting an accurate and high-performance classification algorithm. This motivates us to tackle the challenges faced by the state-of-the-art SBRs classification methods by selecting a high-performance machine learning (i.e., deep learning) algorithm. Therefore, the main goal of this paper is to automate the process of determining which bug report can be labeled as SBR through the use of deep learning techniques. We first extracted 45,940 bug reports from publicly available datasets of five software repositories (e.g., the work of Peters et al. and Shu et al.). Second, we conducted a study on the classification of SBRs using deep learning, where we built a fasttext classifier. We then examined the accuracy of using fasttext in detecting SBRs. Our results show that fasttext can identify SBRs with an average F1 score of 0.81. Furthermore, we investigated the generalizability of identifying SBRs by applying cross-project validation, and our results show that the fasttext classifier achieves an average F1 value of 0.65. Data and results are available at https://github.com/isultane/fasttext_classifications.

show abstract

Predictive Models in Software Engineering: Challenges and Opportunities

Cited by 23 publications

References 455 publications

A research landscape on software defect prediction

A research landscape on software defect prediction

OpenPredict - An Open Research Dataset and Evaluation Protocol for Fine-grained Predictive Testing

A Deep Learning Approach to Improve Classification of Security Bug Reports

Contact Info

Product

Resources

About