Wait for it: identifying “On-Hold” self-admitted technical debt

Abstract

Self-admitted technical debt refers to situations where a software developer knows that their current implementation is not optimal and indicates this using a source code comment. In this work, we hypothesize that it is possible to develop automated techniques to understand a subset of these comments in more detail, and to propose tool support that can help developers manage self-admitted technical debt more effectively. Based on a qualitative study of 333 comments indicating self-admitted technical debt, we first identify one particular class of debt amenable to automated management: on-hold self-admitted technical debt (on-hold SATD), i.e., debt which contains a condition to indicate that a developer is waiting for a certain event or an updated functionality having been implemented elsewhere. We then design and evaluate an automated classifier which can identify these on-hold instances with an area under the receiver operating characteristic curve (AUC) of 0.98 as well as detect the specific conditions that developers are waiting for. Our work presents a first step towards automated tool support that is able to indicate when certain instances of self-admitted technical debt are ready to be addressed.

Characterizing and Mitigating Self-Admitted Technical Debt in Build Systems

“…Classifier preparation. Previous studies [26,27,49] reported that classifiers trained by combining N-gram IDF and auto-sklearn machine learning tend to outperform classifiers that are trained with single word features. Heeding their advice, we train our classifiers using auto-sklearn [12], which automatically determines effective machine learning pipelines for classification.…”

“…An analysis of coded SATD comments revealed that bug report links usually appear in comments. Thus, for all SATD comments, we replace hyperlinks with abstracturl by using the regular expression similar to the previous study [27]: h t t p s ? : \ / \ / (www\ . )…”

“…Additionally, we apply Spacy 13 to lemmatize words, which accounts for term conjugation. Although it is common practice, we opt to exclude stop words removal, since stop words like 'for' and 'until' convey critical semantics in the context of SATD comments [27].…”

“…These classifiers have been broadly adopted in prior studies [18,50]. Similar to prior work [27], we apply TF-IDF [42] to extract the features for our baseline classifiers.…”

Automating the removal of obsolete TODO comments

“…Huang et al [13] used the text-mining based methods to predict whether a comment contains SATD or not. Rungroj et al [21] first introduced the concept of "on-hold" SATD and proposed a tool [20] to identify and remove the "on-hold" SATD automatically. Even if obsolete TODOs and "on-hold" SATD are similar, there are several differences between our work and theirs: (i) According to their definition in [21], the "on-hold" SATD contains a specific condition to be triggered, while the obsolete TODOs in our study do not have such constraint.…”