Generating Diverse Test Suites for Gson Through Adaptive Fitness Function Selection

Almulla, Hussein

doi:10.1007/978-3-030-59762-7_18

Cited by 4 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have previously proposed the concept of adaptive fitness function selection, and demonstrated its potential for increasing the number of discovered exceptions [6]. We also have published a small pilot study for the Gson case examples and the diversity goal [5]. This publication extends both studies in significant ways:…”

Section: Introductionmentioning

confidence: 83%

“…In both the diversity and Strong Mutation Experiments, we use the following 434 faults: Chart (26 faults), Closure (174 faults), Lang (64 faults), Math (106 faults), Mockito (38 faults), and Time (26 faults). In addition, for the diversity goal, we also use the Gson project (18 faults)-which was initially assessed in a pilot study [5]-bringing the total case examples for the diversity experiment to 452.…”

Section: Case Examplesmentioning

confidence: 99%

“…A special class of search-based approaches are known as hyperheuristic, or self-adaptive, approaches [53,6,5]. These approaches incorporate a learning phase in order to automatically tune the search strategy towards particular problem instances [10].…”

Section: Search-based Test Generationmentioning

confidence: 99%

“…Defects4J is a benchmark of real faults extracted from Java projects [55] 5 . For each fault, Defects4J provides access to the faulty and fixed versions of the code, developer-written test cases that expose the fault, and a list of classes and lines of code modified by the patch that fixes the fault.…”

Section: Case Examplesmentioning

confidence: 99%

See 3 more Smart Citations

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

Almulla

2020

2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)

Self Cite

View full text Add to dashboard Cite

Search-based test generation is guided by feedback from one or more fitness functions-scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals-such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage-do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of real Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show small improvements on the third when the number of generations of evolution is fixed. Additionally, for all goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows Evo-SuiteFIT to make strategic choices that efficiently produce more effective test suites, and examining its choices offers insight into how to attain our testing goals. We find that AFFS is a powerful technique to apply when an effective fitness function does not already exist for generating tests to achieve a testing goal.

show abstract

Section: Introductionmentioning

confidence: 83%

Section: Case Examplesmentioning

confidence: 99%

Section: Search-based Test Generationmentioning

confidence: 99%

Section: Case Examplesmentioning

confidence: 99%

See 2 more Smart Citations

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

Almulla

2020

2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many authors use ML to enhance existing test generation approaches-often based on GAs. [5,60] use RL to adapt GA's test generation strategy by selecting the fitness functions optimized by the GA to identify functions that trigger exceptions [5] and input diversity [60]. [62] a method call with one whose return type is a subclass of the original method's, and it can replace a call to a public method with a call to a method that calls a private method.…”

Section: Unit Test Generationmentioning

confidence: 99%

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

Fontes¹,

Gay²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Context: Machine learning (ML) may enable effective automated test generation.Objectives: We characterize emerging research, examining testing practices, researcher goals, ML techniques applied, evaluation, and challenges.Methods: We perform a systematic literature review on a sample of 97 publications.Results: ML generates input for system, GUI, unit, performance, and combinatorial testing or improves the performance of existing generation methods. ML is also used to generate test verdicts, property-based, and expected output oracles. Supervised learning-often based on neural networks-and reinforcement learning-often based on Q-learning-are common, and some publications also employ unsupervised or semi-supervised learning. (Semi-/Un-)Supervised approaches are evaluated using both traditional testing metrics and ML-related metrics (e.g., accuracy), while reinforcement learning is often evaluated using testing metrics tied to the reward function. Conclusion:Work-to-date shows great promise, but there are open challenges regarding training data, retraining, scalability, evaluation complexity, ML algorithms employed-and how they are applied-benchmarks, and replicability. Our findings can serve as a roadmap and inspiration for researchers in this field.

show abstract

The integration of machine learning into automated test generation: A systematic mapping study

Fontes

Gay

2023

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Machine learning (ML) may enable effective automated test generation. We characterize emerging research, examining testing practices, researcher goals, ML techniques applied, evaluation, and challenges in this intersection by performing. We perform a systematic mapping study on a sample of 124 publications. ML generates input for system, GUI, unit, performance, and combinatorial testing or improves the performance of existing generation methods. ML is also used to generate test verdicts, property‐based, and expected output oracles. Supervised learning—often based on neural networks—and reinforcement learning—often based on Q‐learning—are common, and some publications also employ unsupervised or semi‐supervised learning. (Semi‐/Un‐)Supervised approaches are evaluated using both traditional testing metrics and ML‐related metrics (e.g., accuracy), while reinforcement learning is often evaluated using testing metrics tied to the reward function. The work‐to‐date shows great promise, but there are open challenges regarding training data, retraining, scalability, evaluation complexity, ML algorithms employed—and how they are applied—benchmarks, and replicability. Our findings can serve as a roadmap and inspiration for researchers in this field.

show abstract

Generating Diverse Test Suites for Gson Through Adaptive Fitness Function Selection

Cited by 4 publications

References 7 publications

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

The Integration of Machine Learning into Automated Test Generation: A Systematic Literature Review

The integration of machine learning into automated test generation: A systematic mapping study

Contact Info

Product

Resources

About