What to Fix? Distinguishing between Design and Non-design Rules in Automated Tools

Abstract

Abstract-Technical debt-design shortcuts taken to optimize for delivery speed-is a critical part of long-term software costs. Consequently, automatically detecting technical debt is a high priority for software practitioners. Software quality tool vendors have responded to this need by positioning their tools to detect and manage technical debt. While these tools bundle a number of rules, it is hard for users to understand which rules identify design issues, as opposed to syntactic quality. This is important, since previous studies have revealed the most significant technical debt is related to design issues. Other research has focused on comparing these tools on open source projects, but these comparisons have not looked at whether the rules were relevant to design. We conducted an empirical study using a structured categorization approach, and manually classify 466 software quality rules from three industry tools-CAST, SonarQube, and NDepend. We found that most of these rules were easily labeled as either not design (55%) or design (19%). The remainder (26%) resulted in disagreements among the labelers. Our results are a first step in formalizing a definition of a design rule, in order to support automatic detection.

ATDx: Building an Architectural Technical Debt Index

“…In order get further insights into the viability of ATDx, we implemented a prototype of ATDx, built by considering a set of predefined SonarQube rules [Ernst et al, 2017], and a large-scale dataset of over 6.7K software projects. The raw data, analysis scripts, and results of such investigation, accompanied by a companion technical report, is made available online for further consultation.…”

Empirical evaluation of an architectural technical debt index in the context of the Apache and ONAP ecosystems

“…The goal of this phase is to establish the set of architectural rules AR T from SonarQube. As input to this phase, we use a set R SQ of Java-based SonarQube rules that were identified as design rules in a previous research ( Ernst et al, 2017 ). Those rules represent a sound starting set of rules of potential architectural relevance, according to our definition of architectural rule presented in ‘Definitions’.…”

“…To select the architectural rules among the ones presented by Ernst et al (2017) , we carry out a manual inspection of the definition of each single rule. Such inspection is based on the publicly-available official documentation of SonarQube ( https://docs.sonarqube.org/latest/user-guide/rules/ ).…”

“…Then, a third researcher with several years of experience in software engineering takes over by (i) resolving possible conflicts and (ii) reviewing the the final set of architectural rules AR SQ . From the initial set of 72 SonarQube design rules presented by Ernst et al (2017) , we identify 45 architectural rules. As detailed in ‘Phase 3: AR SQ Dataset Building’, we further refine the set of identified rules during Phase 3 by removing the architectural rules which are not included in the SonarQube quality profiles ( https://docs.sonarqube.org/latest/instance-administration/quality-profiles/ ) of the selected software portfolios, leading us to a final set of 25 included rules.…”

A study on the evolution of software quality and technical debt in open source applications

“…In a recent study [56], the authors, being motivated by the perception that design problems are more significant than coding errors for long-term software maintenance, aimed at investigating how three major TD tools (CAST, NDepend, SonarQube) capture design debt. Particularly, the authors distinguished the rules that capture design debt from a total of 466 examined rules from all three tools.…”