Are Static Analysis Violations Really Fixed? A Closer Look at Realistic Usage of SonarQube

Marcílio, Diego; Bonifácio, Rodrigo; Monteiro, Eduardo; Canedo, Edna Dias; Luz, Welder Pinheiro; Pinto, Gustavo

doi:10.1109/icpc.2019.00040

Cited by 71 publications

(67 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…After transforming their data to show the sum of open ASAT warnings for given days it shows rising sums of ASAT warnings for almost all of the projects. Furthermore, current research found that only a small fraction of ASAT warnings are fixed by developers (Liu et al 2018;Marcilio et al 2019;Kim and Ernst 2007b). Table 5 provide us with additional interesting insights.…”

Section: Discussionmentioning

confidence: 85%

“…There is no theory for explaining the continued growth of ASAT warnings but it could be interpreted as an indication that some warnings are ignored by developers because they may be deemed unnecessary or false positives (warnings in code without problems). An increasing number of ASAT warnings is also supported by the raw data provided by (Marcilio et al 2019) in their replication kit. Although Marcilio et al investigate real usage of Sonarcube by developers and primarily the resolution times of ASAT warnings, they provide the dates where ASAT warnings are opened and closed.…”

Section: Discussionmentioning

confidence: 99%

“…However, checking adherence to coding standards via CI quality gates is an industry practice, which is probably a contributing factor. The most frequently fixed warnings found by (Marcilio et al 2019) also contain rules regarding naming conventions and coding style. The only groups of ASAT warnings for which we found more introductions than removals in our trend analysis were import and clone rules.…”

Section: Discussionmentioning

confidence: 99%

“…The topics covered in research with regards to ASATs are concerned with configuration changes (Beller et al 2016), CI-pipelines (Zampetti et al 2017), finding reported defects (Habib and Pradel 2018;Thung et al 2012;Vetro et al 2011) or warning resolution times (Marcilio et al 2019;Digkas et al 2018). Vassallo et al (2019) provide a thorough investigation of developer usage of ASATs in different developing contexts.…”

Section: Introductionmentioning

confidence: 99%

“…One of the problems identified for ASAT usage is the number of false positives (Johnson et al 2013;Christakis and Bird 2016;Vassallo et al 2019) for which warning priorization (Kim and Ernst 2007a;2007b) was proposed, sometimes as actionable warning identification (Heckman and Williams 2009), see also the systematic literature review by Heckman and Williams (2011). Developers perceive ASATs as quality improving (Marcilio et al 2019;Devanbu et al 2016) although the percentage of resolved ASAT warnings vary, e.g., 0.5% (Liu et al 2018), 8.77% (Marcilio et al 2019), 6%-9% (Kim and Ernst 2007b) and 36.3% (Digkas et al 2018).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

2020

View full text Add to dashboard Cite

Automated static analysis tools (ASATs) have become a major part of the software development workflow. Acting on the generated warnings, i.e., changing the code indicated in the warning, should be part of, at latest, the code review phase. Despite this being a best practice in software development, there is still a lack of empirical research regarding the usage of ASATs in the wild. In this work, we want to study ASAT warning trends in software via the example of PMD as an ASAT and its usage in open source projects. We analyzed the commit history of 54 projects (with 112,266 commits in total), taking into account 193 PMD rules and 61 PMD releases. We investigate trends of ASAT warnings over up to 17 years for the selected study subjects regarding changes of warning types, short and long term impact of ASAT use, and changes in warning severities. We found that large global changes in ASAT warnings are mostly due to coding style changes regarding braces and naming conventions. We also found that, surprisingly, the influence of the presence of PMD in the build process of the project on warning removal trends for the number of warnings per lines of code is small and not statistically significant. Regardless, if we consider defect density as a proxy for external quality, we see a positive effect if PMD is present in the build configuration of our study subjects.

show abstract

Section: Discussionmentioning

confidence: 85%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

2020

View full text Add to dashboard Cite

show abstract

Evolution of technical debt remediation in Python: A case study on the Apache Software Ecosystem

Tan

Feitosa

Avgeriou

et al. 2020

J Software Evolu Process

View full text Add to dashboard Cite

In recent years, the evolution of software ecosystems and the detection of technical debt received significant attention by researchers from both industry and academia. While a few studies that analyze various aspects of technical debt evolution already exist, to the best of our knowledge, there is no large‐scale study that focuses on the remediation of technical debt over time in Python projects—that is, one of the most popular programming languages at the moment. In this paper, we analyze the evolution of technical debt in 44 Python open‐source software projects belonging to the Apache Software Foundation. We focus on the type and amount of technical debt that is paid back. The study required the mining of over 60K commits, detailed code analysis on 3.7K system versions, and the analysis of almost 43K fixed issues. The findings show that most of the repayment effort goes into testing, documentation, complexity, and duplication removal. Moreover, more than half of the Python technical debt is short term being repaid in less than 2 months. In particular, the observations that a minority of rules account for the majority of issues fixed and spent effort suggest that addressing those kinds of debt in the future is important for research and practice.

show abstract

Further investigation of the survivability of code technical debt items

Zabardast

Bennin

González-Huerta

2022

J Software Evolu Process

View full text Add to dashboard Cite

Context: Technical debt (TD) discusses the negative impact of sub-optimal decisions to cope with the need-for-speed in software development. Code technical debt items (TDI) are atomic elements of TD that can be observed in code artifacts. Empirical results on open-source systems demonstrated how code-smells, which are just one type of TDIs, are introduced and "survive" during release cycles. However, little is known about whether the results on the survivability of code-smells hold for other types of code TDIs (i.e., bugs and vulnerabilities) and in industrial settings. Goal: Understanding the survivability of code TDIs by conducting an empirical study analyzing two industrial cases and 31 open-source systems from Apache Foundation. Method: We analyzed 133,670 code TDIs (35,703 from the industrial systems)detected by SonarQube (in 193,196 commits) to assess their survivability using survivability models. Results: In general, code TDIs tend to remain and linger for long periods in open-source systems, whereas they are removed faster in industrial systems. Code TDIs that survive over a certain threshold tend to remain much longer, which confirms previous results. Our results also suggest that bugs tend to be removed faster, while code smells and vulnerabilities tend to survive longer.

show abstract

Are Static Analysis Violations Really Fixed? A Closer Look at Realistic Usage of SonarQube

Cited by 71 publications

References 25 publications

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

A longitudinal study of static analysis warning evolution and the effects of PMD on software quality in Apache open source projects

Evolution of technical debt remediation in Python: A case study on the Apache Software Ecosystem

Further investigation of the survivability of code technical debt items

Contact Info

Product

Resources

About