A Suite of Java Message-passing Benchmarks to Support the Validation of Testing Models, Criteria and Tools

Trace Generation and Deterministic Execution for Concurrent Programs

“…Multiple executions of a program using the same test case can exercise different pairs of synchronization being able to even generate different outputs, which are correct or not [Carver and Tai 1991]. This type of behaviour can hide defects, such as unprotected shared data, loss of messages or notifications, deadlocks, starvation, among others [Farchi et al 2003]. Testing activity for concurrent programs is extremely challenging and prone to errors [Bocchino et al 2009], as defects revealed in execution can disappear the next time: this makes the testing difficult and expensive.…”

“…Besides re-executing, this information can be used to evaluate the coverage of synchronization primitives [Souza et al 2014]. Furthermore, it can be used as a basis to generate new synchronization pairs so that the coverage rate [Lei and Carver 2006] can be increased [Sarmanho et al 2008].…”

Structural testing criteria for concurrent programs considering loop execution

“…Given the importance and the lack of concurrent benchmarks for empirical validations, (DOURADO, 2015;DOURADO et al, 2016) presents a benchmark suite of Java messagepassing programs to support testing criteria and tools. Most of the developed benchmarks are relatively small size programs of simple problems, which main focus was the complexity of communications among processes, the implementation for both distributed and shared memory paradigms, the utilization of different communication patterns and test cases with documentation (DOURADO et al, 2016).…”

Um framework para avaliação sistemática de técnicas de teste no contexto de programação concorrente

“…Este software pode ser caracterizado como um exemplo didático (programa exemplo), aplicações industriais ou um conjunto de programas provenientes de benchmarks. Existe uma variedade de benchmarks que podem ser usados como objetos para avaliação experimental na área de teste de software concorrente (DOURADO et al, 2016;MERCER, 2009a;YANG et al, 2008;VALGRIND-DEVELOPERS, 2012).…”

“…This software can be a didactic example with less then 1000 LOC (example programs), real industrial practical applications (real applications) or a set of programs from a benchmark. There are a variety of benchmarks that can be used as objects for empirical evaluation in the concurrent software testing area [97,98,99,100,101]. The majority are derived from benchmarks of programs aiming guarantees control and more replicate results to the experiments.…”