Automating Change-Level Self-Admitted Technical Debt Determination

Yan, Meng; Xia, Xin; Shihab, Emad; Lo, David; Yin, Jianwei; Yang, Xiaohu

doi:10.1109/tse.2018.2831232

Cited by 69 publications

(64 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[13]) 3) Understand the factors that lead to a greater likelihood of defects such as defect prone software components using code metrics (e.g., ratio comment to code, cyclomatic complexity) [27], [64], [65] or process metrics (e.g., number of changes, recent activity) [29], [39], [70], [73]. 4) Use predictors to proactively fix defects [7], [47], [58] 5) Study defect prediction not only just release-level [3], [18] but also change-level or just-in-time [48], [75], [86], [107] both for research and also industry. 6) Explore "transfer learning" where predictors from one project are applied to another [56], [74].…”

Section: Why Study Defect Prediction?mentioning

confidence: 99%

“…Another issue is that all the data-mining process in this paper focused on learning the change level or commit-level as a whole. In this approach, changes attributes on multiple files (from Table 6) are averaged out within a commit [48], [75], [86], [107]. This tend to be language agnostic and the results can be generalized.…”

Section: Future Workmentioning

confidence: 99%

“…This tend to be language agnostic and the results can be generalized. However, that approach might be enhanced by including features extracted within the commit messages (as proposed by Yan et al [107]) or by including file or function attributes (i.e. learning on static code attributes such as C.K.…”

Section: Future Workmentioning

confidence: 99%

See 2 more Smart Citations

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Standard automatic methods for recognizing problematic development commits can be greatly improved via the incremental application of human+artificial expertise. In this approach, called EMBLEM, an AI tool first explore the software development process to label commits that are most problematic. Humans then apply their expertise to check those labels (perhaps resulting in the AI updating the support vectors within their SVM learner). We recommend this human+AI partnership, for several reasons. When a new domain is encountered, EMBLEM can learn better ways to label which comments refer to real problems. Also, in studies with 9 open source software projects, labelling via EMBLEM's incremental application of human+AI is at least an order of magnitude cheaper than existing methods (≈ eight times). Further, EMBLEM is very effective. For the data sets explored here, EMBLEM better labelling methods significantly improved Popt20 and G-score performance in nearly all the projects studied here. TABLE 1This paper argues against using keywords like these as a method for labelling a commit as "buggy'.

show abstract

Section: Why Study Defect Prediction?mentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

See 1 more Smart Citation

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…A prediction model with the best performance provides an AUC value close to 1. The ROC analysis is robust in case of imbalanced class distributions and asymmetric misclassification costs …”

Section: Experiments Design and Results Analysismentioning

confidence: 99%

Multitask defect prediction

Chen

Xia

et al. 2019

J Software Evolu Process

Self Cite

View full text Add to dashboard Cite

Within‐project defect prediction assumes that we have sufficient labeled data from the same project, while cross‐project defect prediction assumes that we have plenty of labeled data from source projects. However, in practice, we might only have limited labeled data from both the source and target projects in some scenarios. In this paper, we want to apply multitask learning to investigate such a new scenario. To our best knowledge, this problem (ie, both the source project and the target project have limited labeled data) has not been thoroughly investigated, and we are the first to propose a novel multitask defect prediction approach mask. mask consists of a differential evolution optimization phase and a multitask learning phase. The former phase aims to find optimal weights for shared and nonshared information in related projects (ie, the target project and its related source projects), while the latter phase builds prediction models for each project simultaneously. To verify the effectiveness of mask, we perform experimental studies on 18 real‐world software projects and compare our approach with four state‐of‐the‐art baseline approaches: single‐task learning (STL), simple combined learning (SCL), Peters filter, and Burak filter. Experimental results show that mask can achieve F1 of 0.397 and AUC of 0.608 on average with a few labeled data (ie, 10% of data). Across the 18 projects, mask can outperform baseline methods significantly in terms of F1 and AUC. Therefore, by utilizing the relatedness among multiple projects, mask can perform significantly better than the state‐of‐the‐art methods. The results confirm that mask is promising for software defect prediction when the source and target projects both have limited training data.

show abstract

“…Technical debt is a term that is used to depict non-optimal choices made in the software development process. Several types of technical debt have been identified including code debt, design and architectural debt, environmental debt, knowledge distribution and documentation debt, and testing debt (Yan et al 2018). A study by Sas and Avgeriou (2019) shows that technical debt degrades maintainability.…”

Section: Introductionmentioning

confidence: 99%

Predicting technical debt from commit contents: reproduction and extension with automated feature selection

Rantala

Mäntylä

2020

Software Qual J

View full text Add to dashboard Cite

Self-admitted technical debt refers to sub-optimal development solutions that are expressed in written code comments or commits. We reproduce and improve on a prior work by Yan et al. (2018) on detecting commits that introduce self-admitted technical debt. We use multiple natural language processing methods: Bag-of-Words, topic modeling, and word embedding vectors. We study 5 open-source projects. Our NLP approach uses logistic Lasso regression from Glmnet to automatically select best predictor words. A manually labeled dataset from prior work that identified self-admitted technical debt from code level commits serves as ground truth. Our approach achieves + 0.15 better area under the ROC curve performance than a prior work, when comparing only commit message features, and + 0.03 better result overall when replacing manually selected features with automatically selected words. In both cases, the improvement was statistically significant (p < 0.0001). Our work has four main contributions, which are comparing different NLP techniques for SATD detection, improved results over previous work, showing how to generate generalizable predictor words when using multiple repositories, and producing a list of words correlating with SATD. As a concrete result, we release a list of the predictor words that correlate positively with SATD, as well as our used datasets and scripts to enable replication studies and to aid in the creation of future classifiers.

show abstract

Automating Change-Level Self-Admitted Technical Debt Determination

Cited by 69 publications

References 69 publications

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

Multitask defect prediction

Predicting technical debt from commit contents: reproduction and extension with automated feature selection

Contact Info

Product

Resources

About