Choosing the fitness function for the job: Automated generation of test suites that detect real faults

Salahirad, Alireza; Almulla, Hussein; Gay, Gregory

doi:10.1002/stvr.1758

Cited by 5 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ability to gain fresh information and knowledge has had a major impact on each of these historical periods. In order to achieve this, we've had to create brand-new economic systems, social revolutions and cultural shifts as well as labour patterns [26]. Wind and water power were heavily utilised in agricultural processes during the agrarian era.…”

Section: Literature Surveymentioning

confidence: 99%

Shift left testing paradigm process implementation for quality of software based on fuzzy

et al. 2023

View full text Add to dashboard Cite

Traditionally, testing is done first at end of the design phase, however this is no longer the case. Testing, finding, and categorising bugs, as well as releasing the development changes into the product, carries a price tag. If the test/verification team discovers a high-severity issue at the end of the lifecycle, the costs may climb. Even if all of the issues are resolved, the release could be delayed. Shift-Left testing is done in isolation by the test/verification team and does not increase testing time, but it has demonstrated to be in sync with product development in some cases. In the context of a process, shifting-left refers to taking action early on. Shift-left testing refers to the practise of testing software earlier in the development cycle than is customary, or to the left in the delivery pipeline, as opposed to the traditional practise of testing software later in the development cycle. Shifting to a "shift left" strategy assumes that the software development team may find bugs faster if they test their code as it is being written, rather than waiting until the end of the project based on fuzzy. Before the code is available for testing, shift left testing encourages developers to write test cases. An agile software development strategy known as "shift left" stresses putting test cases in place early in the life cycle of a project rather than at the conclusion. It also means that automated tests will cover a larger portion of a project's planned functionality rather to just a small portion. The shift left testing adoption benefits the organization to reduce the development cost and time as the testing is done along with development to avoid delay in the process. This paper analyse the benefits of organizations who adopted shift left testing in the software development process.

show abstract

Section: Literature Surveymentioning

confidence: 99%

Shift left testing paradigm process implementation for quality of software based on fuzzy

et al. 2023

View full text Add to dashboard Cite

show abstract

“…[32][33][34][35] Dynamic test generation methodologies like codeless automation and auto generation of tests have been in research and being adopted by the industry professionals. [36][37][38][39] Model-based automated test generation is used for high coverage, and keyword-driven GUI tests are user-friendly. [40][41][42] UML-based test generation for textual and visual patterns for generating tests is used for functional test coverage.…”

Section: Review Of Literaturementioning

confidence: 99%

A survey on software test automation return on investment, in organizations predominantly from Bengaluru, India

Reanzi

Thangaiah

2021

International Journal of Engineering Business Management

View full text Add to dashboard Cite

Software industry has adopted automated testing widely. The most common method adopted is graphical user interface test automation for the functional scenarios to reduce manual testing and increase the repeatability. Reducing execution time and redundancy to achieve quality “go/no go decisions” provides rational for the executive management to allocate funds to adopt automation and invest in the setup including people, process, and tools to achieve faster time to market. There are a variety of practices engaged by testers, like frameworks, tools, methods, procedures, models, and technologies to achieve automation. Nonetheless, the actual effectiveness in terms of return on investment (ROI) is not known, though there are various formulas to calculate ROI of test automation. The factors that determine the ROI are maturity of test automation, purpose, or intent, picking right tests for automation, knowledge of the tester to derive test coverage, domain expertise, defining right metrics like defects found by automation runs, cost versus reduction of time, labor versus quality of scripts, repeatability, and results. These factors form the base of the questions designed for the survey. The paper presents a survey and analysis to understand the ROI of test automation from industry test professionals from both product and services organizations.

show abstract

“…Different techniques were studied to build effective SBST frameworks that proposed the usage of different (a) optimization algorithms (e.g., [23,59]), (b) input types (e.g., [76]), (c) surrogate models (e.g., [67]), and (d) fitness functions (e.g., [62]). We focus on the fitness function design, a challenging task for designing effective SBST frameworks ( [5,7,12,19,[79][80][81]95]).…”

Section: Introductionmentioning

confidence: 99%

“…[44,51,52]). However, despite the breadth and diversity of testing domains and solutions, the fitness functions design is still complex and time-consuming (e.g., [7,9,81]).…”

Section: Introductionmentioning

confidence: 99%

Search-based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions

Formica¹,

Askarpour²,

Menghi³

2022

Preprint

View full text Add to dashboard Cite

Search-based software testing (SBST) typically relies on fitness functions to guide the search exploration toward software failures. There are two main techniques to define fitness functions: (a) automated fitness function computation from the specification of the system requirements and (b) manual fitness function design. Both techniques have advantages. The former uses information from the system requirements to guide the search toward portions of the input domain that are more likely to contain failures. The latter uses the engineers' domain knowledge.We propose ATheNA, a novel SBST framework that combines fitness functions that are automatically generated from requirements specifications and manually defined by engineers. We design and implement ATheNA-S, an instance of ATheNA that targets Simulink ® models. We evaluate ATheNA-S by considering a large set of models and requirements from different domains. We compare our solution with an SBST baseline tool that supports automatically generated fitness functions, and another one that supports manually defined fitness functions. Our results show that ATheNA-S generates more failure-revealing test cases than the baseline tools and that the difference between the performance of ATheNA-S and the baseline tools is not statistically significant. We also assess whether ATheNA-S could generate failure-revealing test cases when applied to a large case study from the automotive domain. Our results show that ATheNA-S successfully revealed a requirement violation in our case study.

show abstract

Choosing the fitness function for the job: Automated generation of test suites that detect real faults

Cited by 5 publications

References 0 publications

Shift left testing paradigm process implementation for quality of software based on fuzzy

Shift left testing paradigm process implementation for quality of software based on fuzzy

A survey on software test automation return on investment, in organizations predominantly from Bengaluru, India

Search-based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions

Contact Info

Product

Resources

About