A public unified bug dataset for java and its assessment regarding metrics and bug prediction

Ferenć, Rudolf; Tóth, Zoltán; Ladányi, Gergely; Siket, István; Gyimóthy, Tibor

doi:10.1007/s11219-020-09515-0

Cited by 37 publications

(34 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We show that our PLS-DA based class level prediction model achieves superior performance compared to the state-of-the-art approaches (i.e. F-measure of 0.44-0.47 at 90% confidence level) when no data re-sampling applied and comparable to others when up-sampling is applied on the largest open bug dataset we know [16,15,17], while training the model is significantly faster, thus finding optimal parameters is much easier. In terms of completeness, which measures the amount of bugs contained in the Java Classes predicted to be defective, PLS-DA outperforms every other algorithm: it found 69.3% and 79.4% of the total bugs with no re-sampling and up-sampling, respectively.…”

Section: Introductionmentioning

confidence: 84%

“…For creating, optimizing, and evaluating our statistical model, we used the Public Unified Bug Dataset for Java [16,15,17]. It contains the data entries of 5 different public bug datasets (PROMISE [45], Eclipse Bug Dataset [56], Bug Prediction Dataset [11], Bugcatchers Bug Dataset [24], and GitHub Bug Dataset [50]) in a unified manner.…”

Section: Dataset and Predictorsmentioning

confidence: 99%

See 1 more Smart Citation

Employing Partial Least Squares Regression with Discriminant Analysis for Bug Prediction

Ferenc,

Siket,

Hegedűs

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Industry 4.0 (including Internet of Things, big data challenge, etc.) as industrial revolution system induces new contents in terms of software quality as well. Forecasting defect proneness of source code has long been a major research concern. Having an estimation of those parts of a software system that most likely contain bugs may help focus testing efforts, reduce costs, and improve product quality.Many prediction models and approaches have been introduced during the past decades that try to forecast bugged code elements based on static source code metrics, change and history metrics, or both. However, there is still no universal best solution to this problem, as most suitable features and models vary from dataset to dataset and depend on the context in which we use them. Therefore, novel approaches and further studies on this topic are highly necessary.In this paper, we employ a chemometric approach -Partial Least Squares with Discriminant Analysis (PLS-DA) -for predicting bug prone Classes in Java programs using static source code metrics. PLS-DA is successfully applied within the field of chemometrics, but to our best knowledge, it has never been used before as a statistical approach in the software maintenance domain for predicting software errors. In addition, we have used rigorous statistical treatments including bootstrap resampling (we also used re-sampling and up-sampling for improving and balancing the training set, resp.) and randomization (permutation) test, and evaluation for representing the software engineering results.We show that our PLS-DA based prediction model achieves superior performances compared to the state-of-the-art approaches (i.e. F-measure of 0.44-0.47 at 90% confidence level) when no data resampling applied and comparable to others when applying up-sampling on the largest open bug dataset, while training the model is significantly faster, thus finding optimal parameters is much easier. In terms of completeness, which measures the amount of bugs contained in the Java Classes predicted to be defective, PLS-DA outperforms every other algorithm: it found 69.3% and 79.4% of the total bugs with no re-sampling and up-sampling, respectively.

show abstract

Section: Introductionmentioning

confidence: 84%

Section: Dataset and Predictorsmentioning

confidence: 99%

Employing Partial Least Squares Regression with Discriminant Analysis for Bug Prediction

Ferenc,

Siket,

Hegedűs

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In order to be able to predict errors in software with different ML techniques, we need a dataset of the right size and quality. The Unified Bug Dataset [Ferenc et al, 2020b] is suitable for this purpose. This dataset merges several datasets, which are the GitHub Bug Dataset [Tóth et al, 2016], the Promise [Jureczko and Madeyski, 2010] dataset, and the Bug Prediction Dataset [D'Ambros et al, 2010].…”

Section: Datasetsmentioning

confidence: 99%

“…We discuss the details of our methodology in Section 3. Finally, we validated the effectiveness of this source code representation to predict bugs on the Unified Bug Dataset [Ferenc et al, 2020b], which is a dataset of buggy and non-buggy classes implemented in Java. We report our results in Section 5, where we also sought answers to our research questions regarding this representation: RQ1 Is there a Doc2Vec parametrization that would produce similar or better results than learning based on code metrics?…”

Section: Introductionmentioning

confidence: 99%

Bug Prediction Using Source Code Embedding Based on Doc2Vec

Aladics¹,

Jász²,

Ferenć³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Bug prediction is a resource demanding task that is hard to automate using static source code analysis. In many fields of computer science, machine learning has proven to be extremely useful in tasks like this, however, for it to work we need a way to use source code as input. We propose a simple, but meaningful representation for source code based on its abstract syntax tree and the Doc2Vec embedding algorithm. This representation maps the source code to a fixed length vector which can be used for various upstream tasks -one of which is bug prediction. We measured this approach's validity by itself and its effectiveness compared to bug prediction based solely on code metrics. We also experimented on numerous machine learning approaches to check the connection between different embedding parameters with different machine learning models. Our results show that this representation provides meaningful information as it improves the bug prediction accuracy in most cases, and is always at least as good as only using code metrics as features.

show abstract

“…SmartSHARK in conjunction with a HPC-Cluster provided us with the means to extract this information for each file in each commit of our candidate projects. OpenStaticAnaylzer is an open sourced version of the commercial tool SourceMeter (FrontEndART 2019) which has been used in multiple studies, e.g., Faragó et al (2015), Szóke et al (2014), and Ferenc et al (2014) and, more recently (Ferenc et al 2020). It works by constructing an Abstract Semantic Graph (ASG) from the source code which is then used to calculate static source code metrics.…”

Section: Metric Extractionmentioning

confidence: 99%

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

2020

View full text Add to dashboard Cite

Automated static analysis tools (ASATs) have become a major part of the software development workflow. Acting on the generated warnings, i.e., changing the code indicated in the warning, should be part of, at latest, the code review phase. Despite this being a best practice in software development, there is still a lack of empirical research regarding the usage of ASATs in the wild. In this work, we want to study ASAT warning trends in software via the example of PMD as an ASAT and its usage in open source projects. We analyzed the commit history of 54 projects (with 112,266 commits in total), taking into account 193 PMD rules and 61 PMD releases. We investigate trends of ASAT warnings over up to 17 years for the selected study subjects regarding changes of warning types, short and long term impact of ASAT use, and changes in warning severities. We found that large global changes in ASAT warnings are mostly due to coding style changes regarding braces and naming conventions. We also found that, surprisingly, the influence of the presence of PMD in the build process of the project on warning removal trends for the number of warnings per lines of code is small and not statistically significant. Regardless, if we consider defect density as a proxy for external quality, we see a positive effect if PMD is present in the build configuration of our study subjects.

show abstract

A public unified bug dataset for java and its assessment regarding metrics and bug prediction

Cited by 37 publications

References 49 publications

Employing Partial Least Squares Regression with Discriminant Analysis for Bug Prediction

Employing Partial Least Squares Regression with Discriminant Analysis for Bug Prediction

Bug Prediction Using Source Code Embedding Based on Doc2Vec

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

Contact Info

Product

Resources

About