Reducing Technical Debt Density: Refactoring vs. Writing Clean New Code

Abstract

Technical Debt (TD) is a powerful metaphor that represents shortcuts taken in a software development project, usually to meet business goals such as limited time or budget. Technical Debt concerns mostly invisible parts of the system (i.e. no visible features or defects) and has a detrimental effect on the maintainability and evolvability of software. Technical Debt manifests in both design and implementation constructs such as sub-optimal design decisions, design and code smells, use of impertinent technologies, lack of documentation etc.Technical Debt cannot realistically be eliminated but it does need to be proactively managed in order to be kept at a sustainable level. Managing Technical Debt is comprised of a number of activities. In this thesis, we focus on two of these activities, which are reported in the literature as the most important: repayment and prevention. As its name suggests, repayment refers to the remediation of Technical Debt by reworking the aforementioned design and implementation constructs; consider for example refactoring the source code, optimizing a design decision or documenting architecture knowledge that has remained tacit. Similarly, prevention of Technical Debt refers to ensuring the underlying issues will not occur in the first place; consider for example, checking for source code violations, supporting design decision making to avoid future maintenance overhead or simply allocating enough resources to follow the right approach instead of the 'quick and dirty' one. In this thesis, we investigate to what extent repayment (through refactoring) and prevention (through writing clean new code) are effective in managing Technical Debt.In order to address the stated problem, we first need to understand better the evolution of technical debt in practice. Specifically we need to investigate whether technical debt grows or remains stable over time, both at the macroscopic level, and at the level of individual issues and their types. Through a large-scale case study, we learned that in the majority of the systems that we studied, there is a significant increase trend on the size, number of issues, and on the complexity metrics of the project. On the other hand, the amount of technical v