MAT: A simple yet strong baseline for identifying self-admitted technical debt

Guo, Zhaoqiang; Liu, Shiran; Liu, Jinping; Li, Yanhui; Chen, Lin; Lü, Hongmin; Zhou, Yuming; Xu, Baowen

doi:10.48550/arxiv.1910.13238

Cited by 3 publications

(11 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2) Easy achieves close to 100% precision (100% on eight projects and 99% on two projects) on identifying the "easy to find" SATDs. These results are higher than the human-derived set of patterns-"todo, fixme, hack, xxx" from Guo et al [3] (MAT). 3) Easy is even more accurate than human experts in identifying the "easy to find" SATDs since 98% of the conflicting comments, which were labeled as Non-SATDs by human but contain the patterns from Easy, are identified as SATDs in our validation study.…”

Section: Introductionmentioning

confidence: 58%

“…A similar study was done by Khamis et al [31]. After analyzing and categorizing comments in source TABLE 1: Differences between our approach and Guo et al [3] How to find patterns for the "easy to find" SATDs How to find the "hard to find" SATDs Guo et al [3] Manually find patterns from the test set. Require large amounts of human effort and results are tested on the same data used for finding those patterns.…”

Section: Machine Learning Approachesmentioning

confidence: 85%

“…As described in §1, we take a two-step approach to identify SATDs: (1) identify patterns for the "easy to find" SATDs with close to 100% precision and automatically classify comments with the patterns as SATDs (without human verification); (2) then apply machine learning techniques to guide human experts to find the rest "hard to find" SATDs with least number of comments read. Interestingly, during the drafting of this paper, we found a preprint [3] that utilized a similar idea to our two-step approach. Guo et al [3] used four keywords ("fixme, todo, hack, xxx") to identify the "easy to find" SATDs and applied supervised learning models to find the rest "hard to find" SATDs.…”

Section: Two-step Approachesmentioning

confidence: 99%

“…Interestingly, during the drafting of this paper, we found a preprint [3] that utilized a similar idea to our two-step approach. Guo et al [3] used four keywords ("fixme, todo, hack, xxx") to identify the "easy to find" SATDs and applied supervised learning models to find the rest "hard to find" SATDs. Although Guo et al consider their approach as just a strong baseline, it still demonstrates the effectiveness of such two-step approaches.…”

Section: Two-step Approachesmentioning

confidence: 99%

“…• Easy: The pattern recognizer of Jitterbug described in Algorithm 2, which iteratively selects the pattern with highest fitness score in (1) until the selected pattern has lower than 80% precision on the training data. • MAT: a baseline approach from Guo et al [3] where a set of human-derived patterns-"todo, fixme, hack, xxx" is applied to find the "easy to find" SATDs. on three different performance metrics • Precision: P recision = T P/(T P + F P ).…”

Section: Rq1: How To Find the Strong Patterns Of The "Easy To Find" S...mentioning

confidence: 99%

See 4 more Smart Citations

Identifying Self-Admitted Technical Debts with Jitterbug: A Two-step Approach

Yu,

Fahid,

et al. 2020

Preprint

View full text Add to dashboard Cite

Keeping track of and managing the self-admitted technical debts (SATDs) is important to maintaining a healthy software project. This requires much time and effort from human experts to identify these SATDs manually. Currently, automated solutions do not have high enough precision and recall in identifying SATDs to fully automate the process. To solve the above problems, we propose a two-step framework called Jitterbug for identifying SATDs by first finding the "easy to find" SATDs automatically with close to 100% precision via a novel pattern recognition technique, then applying machine learning techniques to assist human experts in manually identifying the rest "hard to find" SATDs with reduced human effort. Our simulation studies on ten software projects show that Jitterbug can identify SATDs more efficiently (with less human effort) than the prior state of the art methods.

show abstract

Section: Introductionmentioning

confidence: 58%

Section: Machine Learning Approachesmentioning

confidence: 85%

Section: Two-step Approachesmentioning

confidence: 99%

Section: Two-step Approachesmentioning

confidence: 99%

Section: Rq1: How To Find the Strong Patterns Of The "Easy To Find" S...mentioning

confidence: 99%

See 3 more Smart Citations

Identifying Self-Admitted Technical Debts with Jitterbug: A Two-step Approach

Yu,

Fahid,

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Self-Admitted Technical Debt in the Embedded Systems Industry: An Exploratory Case Study

Soliman

Avgeriou

et al. 2023

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Technical debt denotes shortcuts taken during software development, mostly for the sake of expedience. When such shortcuts are admitted explicitly by developers (e.g., writing a TODO/Fixme comment), they are termed as Self-Admitted Technical Debt or SATD. There has been a fair amount of work studying SATD management in Open Source projects, but SATD in industry is relatively unexplored. At the same time, there is no work focusing on developers' perspectives towards SATD and its management. To address this, we conducted an exploratory case study in cooperation with an industrial partner to study how they think of SATD and how they manage it. Specifically, we collected data by identifying and characterizing SATD in different sources (issues, source code comments, and commits) and carried out a series of interviews with 12 software practitioners. The results show: 1) the core characteristics of SATD in industrial projects; 2) developers' attitudes towards identified SATD and statistics; 3) triggers for practitioners to introduce and repay SATD; 4) relations between SATD in different sources; 5) practices used to manage SATD; 6) challenges and tooling ideas for SATD management.

show abstract

Management of Self-Admitted Technical Debt using Machine Learning

View full text Add to dashboard Cite

Technical debt is a prevalent phenomenon in software development, where developers make sub-optimal decisions to achieve short-term goals, often at the expense of long-term evolvability and maintainability. These decisions, analogous to financial debt, incur principal and interest over time, which can negatively affect various aspects of software development, and especially making changes either for implementing new features or for fixing bugs. Therefore, effective management of technical debt is crucial to ensure software health in the long run.One way to manage technical debt is by identifying and documenting it explicitly within various software artifacts. This category of technical debt is known as Self-Admitted Technical Debt (SATD), where developers use textual artifacts, such as source code comments, commit messages, issue tracking systems, and pull requests, to indicate the presence of technical debt and its possible consequences. For example, a developer may write a code comment like "TODO: this method needs refactoring" to admit that the current code is not optimal and requires improvement in the future. By identifying and analyzing SATD, developers can gain insights into the technical debt landscape of a software system and take appropriate actions to address it.Despite the prevalence and importance of technical debt in software development, current research and practice on managing technical debt, especially SATD, are inadequate. Existing approaches mainly focus on identifying SATD from source code comments, neglecting other textual artifacts that contain valuable information about SATD, such as issues, commit messages, and pull requests. This oversight leads to undetected SATD accumulation, which can potentially spiral out of control and threaten software quality and sustainability. Moreover, code comments often lack essential contextual and repayment information that can help developers understand the rationale, implications, and solutions for SATD. This lack of visibility hinders informed decisions to repay SATD and hides the true status of resolved SATD. Therefore, there is a need for novel and comprehensive approaches to identify, analyze, and repay SATD from various textual artifacts in software projects.To fulfill this objective, as a first step we explore SATD in issue tracking systems before considering multiple sources. By examining SATD in these systems, we can Abstract better comprehend the nature, causes, and repayment of SATD in software projects. To this end, we conducted a qualitative analysis on a sample of 500 issues, identifying and analyzing sentences in issues that refer to SATD. Our research revealed eight types of SATD found in issue trackers: architecture, build, code, defect, design, documentation, requirement, and test debt. We also identified three distinct cases of technical debt identification in issue trackers, with only a small fraction (13.1%) of technical debt being recognized by its creators. Interestingly, the majority of SATD was repaid primarily by those who identi...

show abstract

MAT: A simple yet strong baseline for identifying self-admitted technical debt

Cited by 3 publications

References 55 publications

Identifying Self-Admitted Technical Debts with Jitterbug: A Two-step Approach

Identifying Self-Admitted Technical Debts with Jitterbug: A Two-step Approach

Self-Admitted Technical Debt in the Embedded Systems Industry: An Exploratory Case Study

Management of Self-Admitted Technical Debt using Machine Learning

Contact Info

Product

Resources

About