A Framework for Evaluating the Results of the SZZ Approach for Identifying Bug-Introducing Changes

Costa, Daniel Alencar da; McIntosh, Shane; Shang, Weiyi; Kulesza, Uirá; Coelho, Roberta; Hassan, Ahmed E.

doi:10.1109/tse.2016.2616306

Cited by 157 publications

(134 citation statements)

References 45 publications

Supporting

Mentioning

132

Contrasting

Order By: Relevance

“…The bug-fixing commits in these projects are then mapped to the previous code changes or commits that introduce the bugs. For that purpose, we used Sliwerski, Zimmermann, and Zeller's SZZ's algorithm [26], [52], [91] to work back in time to identify what code changes lead to the bug report (and the SZZ implementation we used here came from Commit.Guru). Rodrguez-Perez et al [84] report that at least 187 papers have also used SZZ in the same way as these experiments of this paper.…”

Section: Datamentioning

confidence: 99%

See 1 more Smart Citation

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Standard automatic methods for recognizing problematic development commits can be greatly improved via the incremental application of human+artificial expertise. In this approach, called EMBLEM, an AI tool first explore the software development process to label commits that are most problematic. Humans then apply their expertise to check those labels (perhaps resulting in the AI updating the support vectors within their SVM learner). We recommend this human+AI partnership, for several reasons. When a new domain is encountered, EMBLEM can learn better ways to label which comments refer to real problems. Also, in studies with 9 open source software projects, labelling via EMBLEM's incremental application of human+AI is at least an order of magnitude cheaper than existing methods (≈ eight times). Further, EMBLEM is very effective. For the data sets explored here, EMBLEM better labelling methods significantly improved Popt20 and G-score performance in nearly all the projects studied here. TABLE 1This paper argues against using keywords like these as a method for labelling a commit as "buggy'.

show abstract

Section: Datamentioning

confidence: 99%

“…The process of linking commits to their associated changed code has been well studied [26], [31], [52], [91], [93], [106]. But once that link is created, it must be labelled as "bug fixing" or otherwise.…”

Section: Introductionmentioning

confidence: 99%

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

2022

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…The available projects can be considered industrial and mature, due to the strict review and inclusion process required by the ASF. Moreover, the included projects have to keep on reviewing their code and follow a strict quality process 6 .…”

Section: Project Selectionmentioning

confidence: 99%

“…The SZZ [15] algorithm tries to identify the fault-inducing commits from a project's version history. The algorithm was developed in 2005 and has since been adopted in more than 200 empirical studies [11], [6].…”

Section: Szz Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

The Technical Debt Dataset

Lenarduzzi

Saarimäki

Taibi

2019

Proceedings of the Fifteenth International Conference on Predictive Models and Data Analytics in Software Engineering

View full text Add to dashboard Cite

Technical Debt analysis is increasing in popularity as nowadays researchers and industry are adopting various tools for static code analysis to evaluate the quality of their code. Despite this, empirical studies on software projects are expensive because of the time needed to analyze the projects. In addition, the results are difficult to compare as studies commonly consider different projects. In this work, we propose the Technical Debt Dataset, a curated set of project measurement data from 33 Java projects from the Apache Software Foundation. In the Technical Debt Dataset, we analyzed all commits from separately defined time frames with SonarQube to collect Technical Debt information and with Ptidej to detect code smells. Moreover, we extracted all available commit information from the git logs, the refactoring applied with Refactoring Miner, and fault information reported in the issue trackers (Jira). Using this information, we executed the SZZ algorithm to identify the fault-inducing and -fixing commits. We analyzed 78K commits from the selected 33 projects, detecting 1.8M SonarQube issues, 38K code smells, 28K faults and 57K refactorings. The project analysis took more than 200 days. In this paper, we describe the data retrieval pipeline together with the tools used for the analysis. The dataset is made available through CSV files and an SQLite database to facilitate queries on the data. The Technical Debt Dataset aims to open up diverse opportunities for Technical Debt research, enabling researchers to compare results on common projects.

show abstract

Empirical study of abnormality in local variables and its application to fault‐prone Java method analysis

Aman

Amasaki

Yokogawa

et al. 2019

J Software Evolu Process

View full text Add to dashboard Cite

Programmers are familiar with local variables, and in many cases, they can freely define the local variables they use. Thus, the properties of these variables are widely diverse, and this may cause variations in the quality of code. Although variables are named in accordance with coding conventions, the following matters have not received much attention from an empirical viewpoint: automatically deciding whether a local variable is “abnormal” and determining the harmful effect of an abnormal variable. This study focuses on the trends in the name, type, and scope of local variables, then proposes the use of the Mahalanobis distance to evaluate their abnormality. The empirical study entailed collecting local variables from eight open‐source software projects, and the paper reports the following findings: (a) the trend in the variation of the names of variables according to their type; (b) the majority of variables have short names with narrow scopes, where a name is often a word or an abbreviation thereof; (c) methods with an abnormal variable are approximately 1.4 times more likely to be fault prone than methods that contain only normal variables; (d) the proposed abnormality metric can be useful in a random forest‐based fault‐prone method analysis model.

show abstract

A Framework for Evaluating the Results of the SZZ Approach for Identifying Bug-Introducing Changes

Cited by 157 publications

References 45 publications

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

The Technical Debt Dataset

Empirical study of abnormality in local variables and its application to fault‐prone Java method analysis

Contact Info

Product

Resources

About