FRUGAL: Unlocking Semi-Supervised Learning for Software Analytics

Tu, Huy

doi:10.1109/ase51524.2021.9678617

Cited by 7 publications

(20 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The gray cells are median values for the corresponding columns. This demonstrates 90% from previous studies [87,95,94,82] goal of this domain is to identify such a static code warning adoptable or unadoptable. The other domains are discussed in the next few sections.…”

Section: Static Code Warningmentioning

confidence: 67%

“…The SE research have limited exposures to semi-supervised learning. For instance, a lot of recent semi-supervised learning works [99,98,83,82,81] within the SE community are mostly classified as one class of SSL, called self-training (such as Yarowsky [96]). Other classes can also include label propagation (such as Zhu and Ghahramani [102]), majority voting (such as Blum and Mitchell [22]), and label spreading (such as Zhou et al [101]).…”

Section: Standard Versus Specialized Methodology For Software Analytics?mentioning

confidence: 99%

“…2. In the initial conference version of this paper [80], some of the data (for the static warning analysis) was recently challenged (in a forthcoming ICSE'22 paper) due to data quality issues. Here, we take the fixed data resulting from that refutation study [50] and repeated all the prior analysis.…”

Section: Contributionsmentioning

confidence: 99%

“…To better support other researchers our scripts and data are on-line at https://github.com/HuyTu7/FRUGAL. [80]. A forthcoming ICSE'22 [49] paper found that the data used for the analysis of #4 has numerous errors.…”

Section: Contributionsmentioning

confidence: 99%

See 3 more Smart Citations

Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics

Tu¹

2023

Preprint

View full text Add to dashboard Cite

In many domains, there are many examples and far fewer labels for those examples; e.g. we may have access to millions of lines of source code, but access to only a handfuls of warnings about that code. In those domains, semi-supervised learners (SSL) can extrapolate labels from a small number of examples to the rest of the data.Standard SSL algorithms use "weak" knowledge (i.e. those not based on specific SE knowledge) such as (e.g.) co-train two learners and use good labels from one to train the other.Another approach of SSL in software analytics is potentially use "strong" knowledge that use SE knowledge. For example, an often-used heuristic in SE is that unusually large artifacts contain undesired properties (e.g. more bugs). This paper argues that such "strong" algorithms perform better than those standard, weaker, SSL algorithms. We show this by learning models from labels generated using weak SSL or our "stronger" FRUGAL algorithm. In four domains (distinguishing security-related bug reports; mitigating bias in decision making; predicting issue close time; and (reducing false alarms in static code warnings), FRUGAL required only 2.5% of the data to be labeled yet out-performed standard semi-supervised learners that relied on (e.g.) some domain independent graph theory concepts.Hence, for future work, we strongly recommend the use of strong heuristics for semi-supervised learning for SE applications. To better support other researchers, our scripts and data are on-line at https://github.com/HuyTu7/FRUGAL.

show abstract

Section: Static Code Warningmentioning

confidence: 67%

Section: Standard Versus Specialized Methodology For Software Analytics?mentioning

confidence: 99%

Section: Contributionsmentioning

confidence: 99%

Section: Contributionsmentioning

confidence: 99%

See 2 more Smart Citations

Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics

Tu¹

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…When changing many options has similar effects, the landscape can be quite smooth and easily mapped out with just a few samples. Hence, many methods [15,35,52,61,63] reason by cluster the data, then explore just a few representatives from each cluster. To say that another way, when the landscape is smooth, it is not necessary to search everywhere.…”

Section: Landscape Analyticsmentioning

confidence: 99%

Optimizing Predictions for Very Small Data Sets: a case study on Open-Source Project Health Prediction

Lustosa¹

2023

Preprint

View full text Add to dashboard Cite

When learning from very small data sets, the resulting models can make many mistakes. For example, consider learning predictors for open source project health. The training data for this task may be very small (e.g. five years of data, collected every month means just 60 rows of training data). Using this data, prior work had unacceptably large errors in their learned predictors.We show that these high errors rates can be tamed by better configuration of the control parameters of the machine learners. For example, we present here a landscape analytics method (called SNEAK) that (a) clusters the data to find the general landscape of the hyperparameters; then (b) explores a few representatives from each part of that landscape.SNEAK is both faster and and more effective than prior state-of-the-art hyperparameter optimization algorithms (FLASH, HYPEROPT, OPTUNA, and differential evolution). For example, SNEAK found its configurations after assessing 10 2 options, while other tools needed to assess 350 to 5000 (for OPTUNA, HYPEROPT) options. More importantly, the configurations found by SNEAK had far less error that other methods. For example, for project health indicators such as 𝐶= number of commits; 𝐼 =number of closed issues, and 𝑅=number of closed pull requests, SNEAK's 12 month prediction errors are {I=0%, R=33% C=47%} while other methods have far larger errors of {I=61%,R=119% C=149%}. We conjecture that SNEAK works so well since it finds the most informative regions of the hyperparameters, then jumps to those regions. Other methods (that do not reflect over the landscape) can waste time exploring less informative options.From this, we make the following conclusions. Firstly, for predicting open source project health, we recommend landscape analytics (e.g. SNEAK). Secondly, and more generally, when learning from very small data sets, using hyperparameter optimization (e.g. SNEAK) to select learning control parameters. Due to its speed and implementation simplicity, we suggest SNEAK might also be useful in other "data-light" SE domains.To assist other researchers in repeating, improving, or even refuting our results, all our scripts and data are available on GitHub at https://github.com/zxcv123456qwe/niSneak.

show abstract

SeDPGK: Semi-supervised software defect prediction with graph representation learning and knowledge distillation

Liu,

Yue,

Chen

et al. 2024

Information and Software Technology

View full text Add to dashboard Cite

FRUGAL: Unlocking Semi-Supervised Learning for Software Analytics

Cited by 7 publications

References 57 publications

Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics

Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics

Optimizing Predictions for Very Small Data Sets: a case study on Open-Source Project Health Prediction

SeDPGK: Semi-supervised software defect prediction with graph representation learning and knowledge distillation

Contact Info

Product

Resources

About