On the Relation between External Software Quality and Static Code Analysis

Plösch, Reinhold; Gruber, H.; Hentschel, Anja; Pomberger, Gustav; Schiffer, Stefan

doi:10.1109/sew.2008.17

Cited by 15 publications

(14 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For practitioners, the most interesting result of this study is that defect density, which we use as a proxy for external software quality, is lower when PMD is included in the buildfile. This is also in line with related research from Plosch et al (2008) who found a positive correlation between defects and the number of ASAT warnings by PMD. Although in our case we are not talking about correlations but differences in reported defects.…”

Section: Discussionsupporting

confidence: 92%

“…Instead, we are primarily interested in general trends regarding ASATs to infer information about the evolution of software quality in our candidate projects. Plosch et al (2008) utilized data collected by (Zimmermann et al 2007) and correlated source code quality metrics and defects with warnings found by different static analysis tools. They used three releases of eclipse and presented correlations for different size, complexity and object oriented source code metrics.…”

Section: Related Workmentioning

confidence: 99%

“…Therefore, the influence of ASATs on defects or software quality as a whole may be more indirect. To the best of our knowledge, only the work by Plosch et al (2008) directly investigates this so far (there is also the work by Rahman et al (2014) however it is not directly investigating correlations). Their work shows a positive correlation between ASAT warnings and defects, although their empirical study is limited to one project.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

2020

View full text Add to dashboard Cite

Automated static analysis tools (ASATs) have become a major part of the software development workflow. Acting on the generated warnings, i.e., changing the code indicated in the warning, should be part of, at latest, the code review phase. Despite this being a best practice in software development, there is still a lack of empirical research regarding the usage of ASATs in the wild. In this work, we want to study ASAT warning trends in software via the example of PMD as an ASAT and its usage in open source projects. We analyzed the commit history of 54 projects (with 112,266 commits in total), taking into account 193 PMD rules and 61 PMD releases. We investigate trends of ASAT warnings over up to 17 years for the selected study subjects regarding changes of warning types, short and long term impact of ASAT use, and changes in warning severities. We found that large global changes in ASAT warnings are mostly due to coding style changes regarding braces and naming conventions. We also found that, surprisingly, the influence of the presence of PMD in the build process of the project on warning removal trends for the number of warnings per lines of code is small and not statistically significant. Regardless, if we consider defect density as a proxy for external quality, we see a positive effect if PMD is present in the build configuration of our study subjects.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

2020

View full text Add to dashboard Cite

show abstract

“…For instance, Plosch et al [19] studied the correlation between the number of FindBugs and PMD issues, and defects in Eclipse SDK 2.0, 2.1 and 3.0. They found positive correlations for both tools (0.34, 0.25 and 0.30 for PMD, and 0.20, 0.08, 0.20 for FindBugs).…”

Section: A First Research Stream: Looking At Single Asa Issues To Fimentioning

confidence: 99%

Investigating Automatic Static Analysis Results to Identify Quality Problems: An Inductive Study

Vetrò

Zazworka

Shull

et al. 2012

2012 35th Annual IEEE Software Engineering Workshop

View full text Add to dashboard Cite

Background: Automatic static analysis (ASA) tools examine source code to discover "issues", i.e. code patterns that are symptoms of bad programming practices and that can lead to defective behavior. Studies in the literature have shown that these tools find defects earlier than other verification activities, but they produce a substantial number of false positive warnings. For this reason, an alternative approach is to use the set of ASA issues to identify defect prone files and components rather than focusing on the individual issues.Aim: We conducted an exploratory study to investigate whether ASA issues can be used as early indicators of faulty files and components and, for the first time, whether they point to a decay of specific software quality attributes, such as maintainability or functionality. Our aim is to understand the critical parameters and feasibility of such an approach to feed into future research on more specific quality and defect prediction models. Method: We analyzed an industrial C# web application using the Resharper ASA tool and explored if significant correlations exist in such a data set.Results: We found promising results when predicting defectprone files. A set of specific Resharper categories are better indicators of faulty files than common software metrics or the collection of issues of all issue categories, and these categories correlate to different software quality attributes.Conclusions: Our advice for future research is to perform analysis on file rather component level and to evaluate the generalizability of categories. We also recommend using larger datasets as we learned that data sparseness can lead to challenges in the proposed analysis process.Automatic static analysis, software quality, defect prediction

show abstract

“…Due to this manual effort, we want to have a closer look on the impact of ASATs on measurable quality. Previous research regarding the impact on quality can be divided into direct measures which target bug fixing commits, e.g., Vetro et al (2011), Thung et al (2012); Habib and Pradel (2018) and indirect measures which use ASAT warnings as part of predictive models, e.g., Nagappan and Ball (2005), Plosch et al (2008), Rahman et al (2014), Querel and Rigby (2018), Lenarduzzi et al (2020), Trautsch et al (2020b) and Querel and Rigby (2021). Both approaches usually suffer from data validation issues.…”

Section: Introductionmentioning

confidence: 99%

Are automated static analysis tools worth it? An investigation into relative warning density and external software quality

Trautsch¹,

Herbold²,

Grabowski³

2021

Preprint

View full text Add to dashboard Cite

Automated Static Analysis Tools (ASATs) are part of software development best practices. ASATs are able to warn developers about potential problems in the code. On the one hand, ASATs are based on best practices so there should be a noticeable effect on software quality. On the other hand, ASATs suffer from false positive warnings, which developers have to inspect and then ignore or mark as invalid. In this article, we ask the question if ASATs have a measurable impact on external software quality, using the example of PMD for Java. We investigate the relationship between ASAT warnings emitted by PMD on defects per change and per file. Our case study includes data for the history of each file as well as the differences between changed files and the project in which they are contained. We investigate whether files that induce a defect have more static analysis warnings than the rest of the project. Moreover, we investigate the impact of two different sets of ASAT rules. We find that, bug inducing files contain less static analysis warnings than other files of the project at that point in time. However, this can be explained by the overall decreasing warning density. When compared with all other changes, we find a statistically significant difference in one metric for all rules and two metrics for a subset of rules. However, the effect size is negligible in all cases, showing that the actual difference in warning density between bug inducing changes and other changes is small at best.

show abstract

On the Relation between External Software Quality and Static Code Analysis

Cited by 15 publications

References 11 publications

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

Investigating Automatic Static Analysis Results to Identify Quality Problems: An Inductive Study

Are automated static analysis tools worth it? An investigation into relative warning density and external software quality

Contact Info

Product

Resources

About