On Parameter Tuning in Search Based Software Engineering: A Replicated Empirical Study

Sayyad, Abdel Salam; Goševa-Popstojanova, Katerina; Ammar, H.H.

doi:10.1109/reser.2013.6

Cited by 32 publications

(27 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This latter contribution may help the engineer to explain and account for optimisation decisions in their discussions with users. In their RESER paper [137] they replicate their previous findings from the ICSE paper that Indicator-Based Evolutionary Algorithm (IBEA) outperforms the well-known Non-Dominated Sorting Genetic Algorithm (NSGA-2). The replication shows that these findings also hold for the SPL feature selection problem, over multiple tuning choices.…”

Section: Search Based Feature Model Selectionsupporting

confidence: 53%

“…Sayyad et al provided a detailed investigation of the multiobjective feature selection problem in three related papers [137,138,140]. In their ICSE paper [140] they studied and evaluated metaheuristic algorithms for the multi-objective SPL feature selection problem.…”

Section: Search Based Feature Model Selectionmentioning

confidence: 99%

“…Velazco's masters thesis [148] presents a comparison of exact and metaheuristic multiobjective approaches to feature selection. Velazco compares the SMT-based exact incremental approach called the Guided Improvement Algorithm with IBEA (which was used by Sayyad et al [137,138,139,140]). While the exact technique can find optimal solutions for small models (with 45 features) in two hours, IBEA scales to larger models with 290 features, which it covers sub-optimally in 20 minutes.…”

Section: Search Based Feature Model Selectionmentioning

confidence: 99%

See 2 more Smart Citations

Search based software engineering for software product line engineering

Harman

Jia

Krinke

et al. 2014

Proceedings of the 18th International Software Product Line Conference - Volume 1

113

View full text Add to dashboard Cite

This paper 1 presents a survey of work on Search Based Software Engineering (SBSE) for Software Product Lines (SPLs). We have attempted to be comprehensive, in the sense that we have sought to include all papers that apply computational search techniques to problems in software product line engineering. Having surveyed the recent explosion in SBSE for SPL research activity, we highlight some directions for future work. We focus on suggestions for the development of recent advances in genetic improvement, showing how these might be exploited by SPL researchers and practitioners: Genetic improvement may grow new products with new functional and non-functional features and graft these into SPLs. It may also merge and parameterise multiple branches to cope with SPL branchmania.

show abstract

Section: Search Based Feature Model Selectionsupporting

confidence: 53%

Section: Search Based Feature Model Selectionmentioning

confidence: 99%

Section: Search Based Feature Model Selectionmentioning

confidence: 99%

See 1 more Smart Citation

Search based software engineering for software product line engineering

Harman

Jia

Krinke

et al. 2014

Proceedings of the 18th International Software Product Line Conference - Volume 1

113

View full text Add to dashboard Cite

show abstract

“…They determined that genetic programming performed consistently well, but was harder to configure. In (Arcuri and Fraser 2011;Sayyad et al 2013), the authors conducted a comprehensive study analyzing the impact of parameter settings in machine learning and software effort estimation. They performed a large study of parameter settings using genetic algorithms.…”

Section: Effort Prediction Approaches Using Genetic Algorithmsmentioning

confidence: 99%

A genetic algorithm based framework for software effort prediction

Murillo-Morera

Quesada-López

Castro-Herrera

et al. 2017

J Softw Eng Res Dev

View full text Add to dashboard Cite

Background: Several prediction models have been proposed in the literature using different techniques obtaining different results in different contexts. The need for accurate effort predictions for projects is one of the most critical and complex issues in the software industry. The automated selection and the combination of techniques in alternative ways could improve the overall accuracy of the prediction models. Objectives: In this study, we validate an automated genetic framework, and then conduct a sensitivity analysis across different genetic configurations. Following is the comparison of the framework with a baseline random guessing and an exhaustive framework. Lastly, we investigate the performance results of the best learning schemes. Methods: In total, six hundred learning schemes that include the combination of eight data preprocessors, five attribute selectors and fifteen modeling techniques represent our search space. The genetic framework, through the elitism technique, selects the best learning schemes automatically. The best learning scheme in this context means the combination of data preprocessing + attribute selection + learning algorithm with the highest coefficient correlation possible. The selected learning schemes are applied to eight datasets extracted from the ISBSG R12 Dataset. Results: The genetic framework performs as good as an exhaustive framework. The analysis of the standardized accuracy (SA) measure revealed that all best learning schemes selected by the genetic framework outperforms the baseline random guessing by 45-80%. The sensitivity analysis confirms the stability between different genetic configurations. Conclusions: The genetic framework is stable, performs better than a random guessing approach, and is as good as an exhaustive framework. Our results confirm previous ones in the field, simple regression techniques with transformations could perform as well as nonlinear techniques, and ensembles of learning machines techniques such as SMO, M5P or M5R could optimize effort predictions.

show abstract

“…Arcuri and Fraser found that setting these parameters to poor values leads to inadequate results [2], and this was reinforced by Sayyad et al [10]. Tuning these parameters to an optimal setting before the algorithm starts is difficult, and most software developers usually lack the knowledge to set these parameters to a suitable value.…”

Section: Introductionmentioning

confidence: 99%

Increasing the efficiency of search-based unit test generation using parameter control

White

2015

Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

View full text Add to dashboard Cite

Automatically generating test suites with high coverage is of great importance to software engineers, but this process is hindered by the vast amount of parameters the tools use to generate tests. Developers usually lack knowledge about the workings of the tools that generate test suites to set the parameters to optimal values, and the optimal values usually change during runtime. Parameter Control automatically adapts parameters during test generation, and has shown to help solve this problem in other areas. To investigate any improvements parameter control could have in search-based generation of test suites, we adapted multiple methods of controlling mutation and crossover rate in EvoSuite, a tool that automatically generates unit test suites. Upon evaluation, clear benefits to controlling parameters were found, but surprisingly, controlling some parameters can sometimes be more harmful to the search than beneficial through increased computation costs.

show abstract

On Parameter Tuning in Search Based Software Engineering: A Replicated Empirical Study

Cited by 32 publications

References 12 publications

Search based software engineering for software product line engineering

Search based software engineering for software product line engineering

A genetic algorithm based framework for software effort prediction

Increasing the efficiency of search-based unit test generation using parameter control

Contact Info

Product

Resources

About