Identifying Self-Admitted Technical Debts With Jitterbug: A Two-Step Approach

Yu, Zhe; Fahid, Fahmid Morshed; Tu, Huy

doi:10.1109/tse.2020.3031401

Cited by 26 publications

(61 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When comparing against the SOTA semi-supervised learning work by Yu et al [84] and the SOTA supervised learning work (with deep learning) by Ren et al [55], our proposed method DebtFree outperforms them significantly. First, DebtFree(100) performs similarly to Ren et al [55]'s work and better than Yu et al [84]'s work while reducing the labeling cost by 2.5 times.…”

Section: Resultmentioning

confidence: 93%

“…Fig. 1: Workflows of DebtFree = Pseudo-Labeling (via Unsupervised Learning, i.e., CLA [47]) + Filtering (via CLA [47] or Jitterbug's Easy [84]) + Active Learning (via Emblem [67], Jitterbug's Hard [84], or this study's Falcon).…”

mentioning

confidence: 99%

“…In this work, we aim to better data generation associated with building models for SATDs identification by reducing the labeling effort come from manual method [67] and improving the labeling quality of fully automated methods [84]. Moreover, our investigation also showed that the effort-aware method we propose, DebtFree, also performs statistically similar or even better than two SOTA works [84,55]. To understand and validate this end-to-end method, DebtFree, we investigate the following research questions: RQ1: How well can the state-of-the-art unsupervised learning method identify SATDs?…”

mentioning

confidence: 99%

“…We investigate variants that stem from Nam et al's unsupervised learning CLA method. As these methods leverage on hidden patterns within the data, we compare them to the pattern-based SOTA for identifying SATDs, also by Yu et al [84].…”

mentioning

confidence: 99%

“…From our exploration of various unsupervised learners, the original CLA by Nam et al performs the best. Moreover, CLA performs similarly to the SOTA pattern-based approach, Easy [84], without having access to the data's labels (100% less effort).…”

mentioning

confidence: 99%

See 4 more Smart Citations

DebtFree: Minimizing Labeling Cost in Self-Admitted Technical Debt Identification using Semi-Supervised Learning

Tu¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Keeping track of and managing Self-Admitted Technical Debts (SATDs) is important for maintaining a healthy software project. Current active-learning SATD recognition tool involves manual inspection of 24% of the test comments on average to reach 90% of the recall. Among all the test comments, about 5% are SATDs. The human experts are then required to read almost a quintuple of the SATD comments which indicates the inefficiency of the tool. Plus, human experts are still prone to error: 95% of the false-positive labels from previous work were actually true positives.To solve the above problems, we propose DebtFree, a two-mode framework based on unsupervised learning for identifying SATDs. In mode1, when the existing training data is unlabeled, DebtFree starts with an unsupervised learner to automatically pseudo-label the programming comments in the training data. In contrasts, in mode2 where labels are available with the corresponding training data, DebtFree starts with a pre-processor that identifies the highly prone SATDs from the test dataset. Then, our machine learning model is employed to assist human experts in manually identifying the remaining SATDs. Our experiments on 10 software projects show that both models yield statistically significant improvement in effectiveness over the state-of-the-art automated and semi-automated models. Specifically, DebtFree can reduce the labeling effort by 99% in mode1 (unlabeled training data), and up to 63% in mode2 (labeled training data) while improving the current active learner's F1 relatively to almost 100%.

show abstract

Section: Resultmentioning

confidence: 93%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

DebtFree: Minimizing Labeling Cost in Self-Admitted Technical Debt Identification using Semi-Supervised Learning

Tu¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Large language model ChatGPT versus small deep learning models for self‐admitted technical debt detection: Why not together?

Li,

Liu

et al. 2024

Softw Pract Exp

View full text Add to dashboard Cite

SummaryGiven the increasing complexity and volume of Self‐Admitted Technical Debts (SATDs), how to efficiently detect them becomes critical in software engineering practice for improving code quality and project efficiency. Although current deep learning methods have achieved good performance in detecting SATDs in code comments, they lack explanation. Large language models such as ChatGPT are increasingly being applied to text classification tasks due to their ability to provide explanations for classification results, but it is unclear how effective ChatGPT is for SATD classification. As the first in‐depth study of ChatGPT for SATD detection, we evaluate ChatGPT's effectiveness, compare it with small deep learning models, and find that ChatGPT performs better on Recall, while small models perform better on Precision. Furthermore, to enhance the performance of these approaches, we propose a novel fusion approach named FSATD which combines ChatGPT with small models for SATD detection so as to provide reliable explanations. Through extensive experiments on 62,276 comments from 10 open‐source projects, we show that FSATD outperforms existing methods in performance of F1‐score in cross‐project scenarios. Additionally, FSATD allows for flexible adjustment of fusion strategies, adapting to different requirements of various application scenarios, and can achieve the best Precision, Recall, or F1‐score.

show abstract

DebtFree: minimizing labeling cost in self-admitted technical debt identification using semi-supervised learning

2022

Empir Software Eng

View full text Add to dashboard Cite

Identifying Self-Admitted Technical Debts With Jitterbug: A Two-Step Approach

Cited by 26 publications

References 46 publications

DebtFree: Minimizing Labeling Cost in Self-Admitted Technical Debt Identification using Semi-Supervised Learning

DebtFree: Minimizing Labeling Cost in Self-Admitted Technical Debt Identification using Semi-Supervised Learning

Large language model ChatGPT versus small deep learning models for self‐admitted technical debt detection: Why not together?

DebtFree: minimizing labeling cost in self-admitted technical debt identification using semi-supervised learning

Contact Info

Product

Resources

About