Static test case prioritization using topic models

Thomas, Stephen W.; Hemmati, Hadi; Hassan, Ahmed E.; Blostein, Dorothea

doi:10.1007/s10664-012-9219-7

Cited by 128 publications

(113 citation statements)

References 52 publications

Supporting

Mentioning

111

Contrasting

Order By: Relevance

“…In our survey of the literature, the following SE tasks have been supported using LDA and all of these papers and approaches used adhoc heuristics to configure LDA, perhaps resulting in suboptimal performance in virtually all the cases: feature location [23], bug localization [6], impact analysis [24], source code labeling [13], aspect identification [14], expert identification [25], software traceability [9], [26], test case prioritization [27], and evolution analysis [28], [16].…”

Section: B Lda Applications To Software Engineeringmentioning

confidence: 99%

How to effectively use topic models for software engineering tasks? An approach based on Genetic Algorithms

Panichella

Dit

Oliveto

et al. 2013

2013 35th International Conference on Software Engineering (ICSE)

229

View full text Add to dashboard Cite

Abstract-Information Retrieval (IR) methods, and in particular topic models, have recently been used to support essential software engineering (SE) tasks, by enabling software textual retrieval and analysis. In all these approaches, topic models have been used on software artifacts in a similar manner as they were used on natural language documents (e.g., using the same settings and parameters) because the underlying assumption was that source code and natural language documents are similar. However, applying topic models on software data using the same settings as for natural language text did not always produce the expected results.Recent research investigated this assumption and showed that source code is much more repetitive and predictable as compared to the natural language text. Our paper builds on this new fundamental finding and proposes a novel solution to adapt, configure and effectively use a topic modeling technique, namely Latent Dirichlet Allocation (LDA), to achieve better (acceptable) performance across various SE tasks. Our paper introduces a novel solution called LDA-GA, which uses Genetic Algorithms (GA) to determine a near-optimal configuration for LDA in the context of three different SE tasks: (1) traceability link recovery, (2) feature location, and (3) software artifact labeling. The results of our empirical studies demonstrate that LDA-GA is able to identify robust LDA configurations, which lead to a higher accuracy on all the datasets for these SE tasks as compared to previously published results, heuristics, and the results of a combinatorial search.

show abstract

Section: B Lda Applications To Software Engineeringmentioning

confidence: 99%

How to effectively use topic models for software engineering tasks? An approach based on Genetic Algorithms

Panichella

Dit

Oliveto

et al. 2013

2013 35th International Conference on Software Engineering (ICSE)

229

View full text Add to dashboard Cite

show abstract

“…We complemented the statistical test by employing a non-parametric effect size measure called Vargha and Delaney's A 12 measure (Vargha and Delaney 2000) to measure the level of differences between two populations. We choose Vargha and Delaney's A 12 measure because it is robust with respect to the shape of the distributions being compared (Thomas et al 2014). Put it another way, it does not require the two populations under comparison to be normally distributed, which is the case in our results of the tools' F1 scores.…”

Section: Rq3: Normalisation By Decompilationmentioning

confidence: 99%

A comparison of code similarity analysers

2017

View full text Add to dashboard Cite

Copying and pasting of source code is a common activity in software engineering. Often, the code is not copied as it is and it may be modified for various purposes; e.g. refactoring, bug fixing, or even software plagiarism. These code modifications could affect the performance of code similarity analysers including code clone and plagiarism detectors to some certain degree. We are interested in two types of code modification in this study: pervasive modifications, i.e. transformations that may have a global effect, and local modifications, i.e. code changes that are contained in a single method or code block. We evaluate 30 code similarity detection techniques and tools using five experimental scenarios for Java source code. These are (1) pervasively modified code, created with tools for source code and bytecode obfuscation, and boiler-plate code, (2) source code normalisation through compilation and decompilation using different decompilers, (3) reuse of optimal configurations over different data sets, (4) tool evaluation using ranked-based measures, and (5) local + global code modifications. Our experimental results show that in the presence of pervasive modifications, some of the general textual similarity measures can offer similar performance to specialised code similarity tools, whilst in the presence of boiler-plate code, highly specialised source code similarity detection techniques and tools outperform textual similarity measures. Our study strongly validates the use of compilation/decompilation as a normalisation technique. Its use reduced false classifications to zero for three of the tools. Moreover, we demonstrate that optimal configurations are very sensitive to a specific data set. After directly applying optimal configurations derived from one data set to another, the tools perform poorly on the new data set. The code similarity analysers are thoroughly evaluated not only based on several well-known pair-based and query-based error measures but also on each specific type of pervasive code modification. This broad, thorough study is the largest in existence and potentially an invaluable guide for future users of similarity detection in source code.

show abstract

“…Similarity-based algorithms have also been applied to regression test case prioritization, based on the distances between pair-wise test cases (Wang et al 2015), using historical failure data (Noor and Hemmati 2015) or topics models (Thomas et al 2014). Research effort have also been devoted to similarity-based selection techniques, in particular, in the model-based domain (Hemmati et al 2013).…”

Section: Previous Workmentioning

confidence: 99%

Similarity-based prioritization of test case automation

et al. 2018

View full text Add to dashboard Cite

The importance of efficient software testing procedures is driven by an ever increasing system complexity as well as global competition. In the particular case of manual test cases at the system integration level, where thousands of test cases may be executed before release, time must be well spent in order to test the system as completely and as efficiently as possible. Automating a subset of the manual test cases, i.e, translating the manual instructions to automatically executable code, is one way of decreasing the test effort. It is further common that test cases exhibit similarities, which can be exploited through reuse when automating a test suite. In this paper, we investigate the potential for reducing test effort by ordering the test cases before such automation, given that we can reuse already automated parts of test cases. In our analysis, we investigate several approaches for prioritization in a case study at a large Swedish vehicular manufacturer. The study analyzes the effects with respect to test effort, on four projects with a total of 3919 integration test cases constituting 35,180 test steps, written in natural language. The results show that for the four projects considered, the difference in expected manual effort between the best and the worst order found is on average 12 percentage points. The results also show that our proposed prioritization method is nearly as good as more resource demanding meta-heuristic approaches Software Qual J at a fraction of the computational time. Based on our results, we conclude that the order of automation is important when the set of test cases contain similar steps (instructions) that cannot be removed, but are possible to reuse. More precisely, the order is important with respect to how quickly the manual test execution effort decreases for a set of test cases that are being automated.

show abstract

Static test case prioritization using topic models

Cited by 128 publications

References 52 publications

How to effectively use topic models for software engineering tasks? An approach based on Genetic Algorithms

How to effectively use topic models for software engineering tasks? An approach based on Genetic Algorithms

A comparison of code similarity analysers

Similarity-based prioritization of test case automation

Contact Info

Product

Resources

About