An evaluation of Monte Carlo-based hyper-heuristic for interaction testing of industrial embedded software applications

Ahmed, Bestoun S.; Enoiu, Eduard Paul; Afzal, Wasif; Zamli, Kamal Z.

doi:10.1007/s00500-020-04769-z

Cited by 16 publications

(7 citation statements)

References 71 publications

(137 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Din and Zamli use Exponential Monte Carlo with Counter (EMCO) as a hyperheuristic to select a low-level heuristic in CIT (Din and Zamli 2018). Ahmed et al (2020) compare EMCO against an improved version using Q-learning, called Q-EMCO, to select the best operator based on historical information.…”

Section: Hyperheuristics In Search-based Software Testingmentioning

confidence: 99%

Learning how to search: generating effective test cases through adaptive fitness function selection

Almulla

2022

Empir Software Eng

View full text Add to dashboard Cite

Search-based test generation is guided by feedback from one or more fitness functions—scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals—such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage—do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show limited improvements on the third when the number of generations of evolution is fixed. Additionally, for two of the three goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows strategic choices that efficiently produce more effective test suites, and examining these choices offers insight into how to attain our testing goals. We find that adaptive fitness function selection is a powerful technique to apply when an effective fitness function does not already exist for achieving a testing goal.

show abstract

Section: Hyperheuristics In Search-based Software Testingmentioning

confidence: 99%

Learning how to search: generating effective test cases through adaptive fitness function selection

Almulla

2022

Empir Software Eng

View full text Add to dashboard Cite

show abstract

“…• MASP as High-Level Hybrid (MASP-HLH) MASP-HLH implementation can also be in the form of a relay (i.e., each meta-heuristic algorithm is sequentially applied) or cooperative (with no given ordering) applied to the same population. MASP-HLH can often be associated with hyper-heuristic algorithms (termed as (meta)-heuristic to choose (meta)-heuristic) [40][41][42][43]. By using many (meta)-heuristic algorithms (or their associated search operators), the hyper-heuristic methodology can be considered as a form of hybridization.…”

Section: Figure 2 Population-based Meta-heuristic Algorithm Implementationsmentioning

confidence: 99%

“…Concerning cooperative MASP-HLH, the work of Ahmad et al in [41], and Zamli et al in [51] and in [42] can be highlighted as relevant examples. Ahmad et al [41] propose a Monte Carlobased hyper-heuristic technique that embeds the Q-learning framework as an adaptive metaheuristic selection and acceptance mechanism. The work adopts low-level search operations from the Cuckoo Search Algorithm (CSA) [33], Jaya Algorithm (JA) [9] and Flower Pollination Algorithm (FPA) [9].…”

Section: Figure 2 Population-based Meta-heuristic Algorithm Implementationsmentioning

confidence: 99%

Section: Figure 2 Population-based Meta-heuristic Algorithm Implement...mentioning

confidence: 99%

See 1 more Smart Citation

Hybrid Henry gas solubility optimization algorithm with dynamic cluster-to-algorithm mapping

Zamli

Kader

Azad

et al. 2021

Neural Comput & Applic

Self Cite

View full text Add to dashboard Cite

Henry Gas Solubility Optimization Algorithm (HGSO) is a recently developed population-based metaheuristic algorithm in the literature. One notable feature of HGSO is that the algorithm divides its (single) population into a set of clusters that are individually mapped to an independent HGSO with its parameter settings (as well as its local best). At a glance, having multiple independent HGSO serving the given clusters in the population can definitely boost exploration (i.e., in terms of roaming the new potential region in the search space for better solution alternatives). However, a closer look reveals two main limitations. Firstly, HGSO-to-cluster mapping is statically defined. To be specific, the defined HGSO-to-cluster mapping does not consider its adaptive performance for the subsequent iteration. Secondly, HGSO implementation ignores the opportunity for hybridization with other meta-heuristic algorithms. With hybridization, one can compensate the limitation of a host algorithm with other algorithms' strength. Best results in the literature have often been associated with hybridization. Addressing these limitations, this paper proposes the development of Hybrid HGSO (HHGSO). Taking HGSO as the host algorithm, HHGSO is hybridized with four recently developed meta-heuristic algorithms, including Jaya Algorithm (JA), Sooty Tern Optimization Algorithm (STOA), Butterfly Optimization Algorithm (BOA) and Owl Search Algorithm (OSA). The individual mapping of each algorithm is made dynamic based on penalized and reward adaptive probability. Comparative performance of HHGSO with the aforementioned algorithms is conducted with a well-known Searchbased Software Engineering (SBSE) problem involving team formation problem. Additionally, the defined hybridization approach has also been adopted as a hybridization template for solving the combinatorial test generation problem with the same meta-heuristic algorithm combinations. Comparative performance is also undertaken against recently developed hyper-heuristic algorithms involving Exponential Monte Carlo with Counter, Modified Choice Function, Improvement Selection Rules, and Fuzzy Inference Selection. Our results indicate that the HHGSO hybridization has usefully improved the performance of the original HGSO and gives superior performance against the given competing algorithms.

show abstract

“…Din and Zamli use Exponential Monte Carlo with Counter (EMCO) as a hyperheuristic to select a low-level heuristic in CIT [21]. Ahmed et al [1] compare EMCO against an improved version using Q-learning, called Q-EMCO, to select the best operator based on historical information.…”

Section: Hyperheuristics In Search-based Software Testingmentioning

confidence: 99%

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

Almulla

2020

2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)

View full text Add to dashboard Cite

Search-based test generation is guided by feedback from one or more fitness functions-scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals-such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage-do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of real Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show small improvements on the third when the number of generations of evolution is fixed. Additionally, for all goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows Evo-SuiteFIT to make strategic choices that efficiently produce more effective test suites, and examining its choices offers insight into how to attain our testing goals. We find that AFFS is a powerful technique to apply when an effective fitness function does not already exist for generating tests to achieve a testing goal.

show abstract

An evaluation of Monte Carlo-based hyper-heuristic for interaction testing of industrial embedded software applications

Cited by 16 publications

References 71 publications

Learning how to search: generating effective test cases through adaptive fitness function selection

Learning how to search: generating effective test cases through adaptive fitness function selection

Hybrid Henry gas solubility optimization algorithm with dynamic cluster-to-algorithm mapping

Learning How to Search: Generating Exception-Triggering Tests Through Adaptive Fitness Function Selection

Contact Info

Product

Resources

About