Better software analytics via “DUO”: Data mining algorithms using/used-by optimizers

Agrawal, Amritanshu; Minku, Leandro L.; Wagner, Markus; Yu, Zhe

doi:10.1007/s10664-020-09808-9

Cited by 25 publications

(13 citation statements)

References 123 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 6 provides an overview of the performance of the 17 configurations when run across all corpora. 6 As we can see, a per-corpus configuration is necessary to achieve the lowest perplexity values in topic modelling ( Figure 6). Many configuration corpora can be optimised (within 5%) with a large number of configurations (Figure 7, red), however, a particular cluster of Stack Overflow corpora requires specialised configurations.…”

Section: Per-corpus Configurationmentioning

confidence: 97%

Predicting Good Configurations for GitHub and Stack Overflow Topic Models

Wagner¹

2019

2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR)

Self Cite

View full text Add to dashboard Cite

Software repositories contain large amounts of textual data, ranging from source code comments and issue descriptions to questions, answers, and comments on Stack Overflow. To make sense of this textual data, topic modelling is frequently used as a text-mining tool for the discovery of hidden semantic structures in text bodies. Latent Dirichlet allocation (LDA) is a commonly used topic model that aims to explain the structure of a corpus by grouping texts. LDA requires multiple parameters to work well, and there are only rough and sometimes conflicting guidelines available on how these parameters should be set. In this paper, we contribute (i) a broad study of parameters to arrive at good local optima for GitHub and Stack Overflow text corpora, (ii) an a-posteriori characterisation of text corpora related to eight programming languages, and (iii) an analysis of corpus feature importance via per-corpus LDA configuration. We find that (1) popular rules of thumb for topic modelling parameter configuration are not applicable to the corpora used in our experiments, (2) corpora sampled from GitHub and Stack Overflow have different characteristics and require different configurations to achieve good model fit, and (3) we can predict good configurations for unseen corpora reliably. These findings support researchers and practitioners in efficiently determining suitable configurations for topic modelling when analysing textual data contained in software repositories.

show abstract

Section: Per-corpus Configurationmentioning

confidence: 97%

Predicting Good Configurations for GitHub and Stack Overflow Topic Models

Wagner¹

2019

2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Note that applications of hyperparameter optimization to software engineering is a very large topic. Elsewhere [4] we offer an extensive literature review on hyperparameter optimization and its applications in software engineering. Here, we offer some overview notes.…”

Section: Related Workmentioning

confidence: 99%

How to “DODGE” Complex Software Analytics

Agrawal

Wei

Chen

et al. 2021

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

Machine learning techniques applied to software engineering tasks can be improved by hyperparameter optimization, i.e., automatic tools that find good settings for a learner's control parameters. We show that such hyperparameter optimization can be unnecessarily slow, particularly when the optimizers waste time exploring "redundant tunings", i.e., pairs of tunings which lead to indistinguishable results. By ignoring redundant tunings, DODGE(E), a tuning tool, runs orders of magnitude faster, while also generating learners with more accurate predictions than seen in prior state-of-the-art approaches.

show abstract

“…There are other indicator such as inverted generational distance [6] (IGD) and hypervolume [14] (HV). However, prior research suggests GD as a more suitable metric to uniformly reflect the overall quality of the solution set [1]. For example, if one intentionally adds very poor solutions into the solution set, IGD and HV cannot reflect the change in the overall quality of the new solution set.…”

Section: Performance Criteriamentioning

confidence: 99%

VEER: Enhancing the Interpretability of Model-based Optimizations

Peng¹,

Kaltenecker²,

Siegmund³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Software comes with many configuration options, satisfying varying needs from users. Exploring those options for non-functional requirements can be tedious, time consuming, and even error-prone (if done manually). Worse, many software systems can be tuned to multiple objectives (e.g., faster response time, fewer memory requirements, decreased network traffic, decreased energy consumption, etc.). Learning how to adjust the system among these multiple objectives is complicated due to the trade-off among objectives; i.e., things that seem useful to achieve one objective could be detrimental to another objective. Consequentially, the optimizer built for one objective may have different (or even opposite) insights on how to locate good solutions from the optimizer built from another objective. In this paper, we define this scenario as the model disagreement problem.One possible solution to this problem is to find a one-dimensional approximation to the N-objective space. In this way, the case is converted to a single-objective optimization, which is naturally confusion-free. This paper demonstrates VEER, a tool that builds such an approximation by combining our dimensionality-reduction heuristic on top of one of the state-of-the-art optimizers, FLASH. VEER can explore very large configuration spaces by evaluating just a small fraction of the total number of configurations (e.g., a space of 81,000 configurations can be explored by 70 samples).The experimental result in this paper demonstrates the feasibility of our approach in terms of the on-par quality of the solution set generated by the optimizer, and the resolved model disagreement within the optimizer. Moreover, we demonstrate that VEER has an improved computational complexity compared to the original optimizer (by up to 1,000 times faster while maintaining on-par performance).

show abstract

Better software analytics via “DUO”: Data mining algorithms using/used-by optimizers

Cited by 25 publications

References 123 publications

Predicting Good Configurations for GitHub and Stack Overflow Topic Models

Predicting Good Configurations for GitHub and Stack Overflow Topic Models

How to “DODGE” Complex Software Analytics

VEER: Enhancing the Interpretability of Model-based Optimizations

Contact Info

Product

Resources

About