Past, Present, and Future of Analyzing Software Data

Bird, Christian; Zimmermann, Thomas

doi:10.1016/b978-0-12-411519-4.00001-x

Cited by 39 publications

(40 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table IV, the precision and recall of the crowd on the 20 original tasks is 80% and 74% respectively (so F 1 ≈ 77%). Based on prior work with data mining from software engineering data [48], [49], we find that these values represent a close correspondence between the TDH results and those from the crowd, hence indicating that the crowd can indeed perform the tasks with high quality.…”

Section: A Rq1: Can the Crowd Reproduce Prior Results With High Qualsupporting

confidence: 54%

Replication Can Improve Prior Results: A GitHub Study of Pull Request Acceptance

Chen

Stolee

2019

2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)

Self Cite

View full text Add to dashboard Cite

Crowdsourcing and data mining can be used to effectively reduce the effort associated with the partial replication and enhancement of qualitative studies.For example, in a primary study, other researchers explored factors influencing the fate of GitHub pull requests using an extensive qualitative analysis of 20 pull requests. Guided by their findings, we mapped some of their qualitative insights onto quantitative questions. To determine how well their findings generalize, we collected much more data (170 additional pull requests from 142 GitHub projects). Using crowdsourcing, that data was augmented with subjective qualitative human opinions about how pull requests extended the original issue. The crowd's answers were then combined with quantitative features and, using data mining, used to build a predictor for whether code would be merged. That predictor was far more accurate that one built from the primary study's qualitative factors (F1=90 vs 68%), illustrating the value of a mixed-methods approach and replication to improve prior results.To test the generality of this approach, the next step in future work is to conduct other studies that extend qualitative studies with crowdsourcing and data mining.

show abstract

Section: A Rq1: Can the Crowd Reproduce Prior Results With High Qualsupporting

confidence: 54%

Replication Can Improve Prior Results: A GitHub Study of Pull Request Acceptance

Chen

Stolee

2019

2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are four hyper-parameters to set when using the Gibbs sampling generative model for LDA [28,7]:…”

Section: Background and Related Workmentioning

confidence: 99%

A Systematic Comparison of Search Algorithms for Topic Modelling—A Study on Duplicate Bug Report Identification

Panichella

2019

Search-Based Software Engineering

View full text Add to dashboard Cite

Latent Dirichlet Allocation (LDA) has been used to support many software engineering tasks. Previous studies showed that default settings lead to sub-optimal topic modeling with a dramatic impact on the performance of such approaches in terms of precision and recall. For this reason, researchers used search algorithms (e.g., genetic algorithms) to automatically configure topic models in an unsupervised fashion. While previous work showed the ability of individual search algorithms in finding near-optimal configurations, it is not clear to what extent the choice of the meta-heuristic matters for SE tasks. In this paper, we present a systematic comparison of five different meta-heuristics to configure LDA in the context of duplicate bug reports identification. The results show that (1) no master algorithm outperforms the others for all software projects, (2) random search and PSO are the least effective meta-heuristics. Finally, the running time strongly depends on the computational complexity of LDA while the internal complexity of the search algorithms plays a negligible role.

show abstract

“…Thus a software quality tool has a time-series focus, and pays more attention to code-smells, idioms, and design-related issues. These tools fall into the software analytics space [11], an emerging research area that seeks to leverage the large volumes of data collected as part of the development process (e.g., lines of code, bugs closed, team structure and activity). No "tools" beside IDEs provide compiler level assistance (Eclipse, for example, flags unknown Java types).…”

Section: B State Of the Practicementioning

confidence: 99%

What to Fix? Distinguishing between Design and Non-design Rules in Automated Tools

Ernst

Bellomo

Özkaya

et al. 2017

2017 IEEE International Conference on Software Architecture (ICSA)

View full text Add to dashboard Cite

Abstract-Technical debt-design shortcuts taken to optimize for delivery speed-is a critical part of long-term software costs. Consequently, automatically detecting technical debt is a high priority for software practitioners. Software quality tool vendors have responded to this need by positioning their tools to detect and manage technical debt. While these tools bundle a number of rules, it is hard for users to understand which rules identify design issues, as opposed to syntactic quality. This is important, since previous studies have revealed the most significant technical debt is related to design issues. Other research has focused on comparing these tools on open source projects, but these comparisons have not looked at whether the rules were relevant to design. We conducted an empirical study using a structured categorization approach, and manually classify 466 software quality rules from three industry tools-CAST, SonarQube, and NDepend. We found that most of these rules were easily labeled as either not design (55%) or design (19%). The remainder (26%) resulted in disagreements among the labelers. Our results are a first step in formalizing a definition of a design rule, in order to support automatic detection.

show abstract

Past, Present, and Future of Analyzing Software Data

Cited by 39 publications

References 11 publications

Replication Can Improve Prior Results: A GitHub Study of Pull Request Acceptance

Replication Can Improve Prior Results: A GitHub Study of Pull Request Acceptance

A Systematic Comparison of Search Algorithms for Topic Modelling—A Study on Duplicate Bug Report Identification

What to Fix? Distinguishing between Design and Non-design Rules in Automated Tools

Contact Info

Product

Resources

About