Vision: bias in systematic grammar-based test suite construction algorithms

Rossouw, Christoff; Fischer, Bernd

doi:10.1145/3486608.3486902

Cited by 1 publication

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Modern approaches like Grimoire [9], or Gramatron [41], use fuzz testing to learn the input structure and then apply structure dependent mutations to create inputs. Recently, Rossouw and Fischer [39] studied the limitations introduced by grammar-based test suite construction methods, showing how they significantly bias test suites for large and real-world programs by favouring some production rules and nonterminals in a CFG over others. June reduces this problem, as the SafeString forces randomly generated input to match local requirements, without relying on high level (and usually more complex) CFGs at some point removed from the point of use.…”

Section: Related Workmentioning

confidence: 99%

June: A Type Testability Transformation for Improved ATG Performance

Bruce

Kelly

Menéndez

et al. 2023

Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Strings are universal containers: they are flexible to use, abundant in code, and difficult to test. String-controlled programs are programs that make branching decisions based on string input. Automatically generating valid test inputs for these programs considering only character sequences rather than any underlying string-encoded structures, can be prohibitively expensive.We present June, a tool that enables Java developers to expose any present latent string structure to test generation tools. June is an annotation-driven testability transformation and an extensible library, JuneLib, of structured string definitions. The core JuneLib definitions are empirically derived and provide templates for all structured strings in our test set.June takes lightly annotated source code and injects code that permits an automated test generator (ATG) to focus on the creation of mutable substrings inside a structured string. Using June costs the developer little, with an average of 2.1 annotations per stringcontrolled class. June uses standard Java build tools and therefore deploys seamlessly within a Java project.By feeding string structure information to an ATG tool, June dramatically reduces wasted effort; branches are effortlessly covered that would otherwise be extremely difficult, or impossible, to cover. This waste reduction both increases and speeds coverage. EvoSuite, for example, achieves the same coverage on June-ed classes in 1 minute, on average, as it does in 9 minutes on the un-June-ed class. These gains increase over time. On our corpus, June-ing a program compresses 24 hours of execution time into ca. 2 hours. We show that many ATG tools can reuse the same June-ed code: a few June annotations, a one-off cost, benefit many different testing regimes.

show abstract

Section: Related Workmentioning

confidence: 99%