Coverage Metrics for Saturation-Based and Search-Based Testing of Concurrent Software

Křena, Bohuslav; Letko, Zdeněk; Vojnar, Tomáš

doi:10.1007/978-3-642-29860-8_14

Cited by 18 publications

(15 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lu et al propose a hierarchy of seven schedule coverage criteria and theoretically analyze their cost [35]. Krena et al propose saturation-based coverage metrics that are derived from dynamic analyses to find concurrency errors [30]. To identify threads across multiple executions, they compute a hash value from the type of a thread and from the identifiers of the first n methods executed in the thread.…”

Section: B Coverage Of Concurrent Programsmentioning

confidence: 99%

Bita: Coverage-guided, automatic testing of actor programs

Tasharofi

Pradel

Lin

et al. 2013

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

Abstract-Actor programs are concurrent programs where concurrent entities communicate asynchronously by exchanging messages. Testing actor programs is challenging because the order of message receives depends on the non-deterministic scheduler and because exploring all schedules does not scale to large programs. This paper presents Bita, a scalable, automatic approach for testing non-deterministic behavior of actor programs. The key idea is to generate and explore schedules that are likely to reveal concurrency bugs because these schedules increase the schedule coverage. We present three schedule coverage criteria for actor programs, an algorithm to generate feasible schedules that increase coverage, and a technique to force a program to comply with a schedule. Applying Bita to real-world actor programs implemented in Scala reveals eight previously unknown concurrency bugs, of which six have already been fixed by the developers. Furthermore, we show our approach to find bugs 122x faster than random scheduling, on average.

show abstract

Section: B Coverage Of Concurrent Programsmentioning

confidence: 99%

Bita: Coverage-guided, automatic testing of actor programs

Tasharofi

Pradel

Lin

et al. 2013

2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE)

View full text Add to dashboard Cite

show abstract

“…As future work, other concurrent coverage criteria, e.g., [11], [13], [14] can be incorporated within the framework to characterize the test case diversity for multithreaded programs. From the experience in [4], domainspecific knowledge can be taken into account to further improve the inherent overhead in test input selection for multithreaded programs.…”

Section: Discussionmentioning

confidence: 99%

“…At Step 6, d(t * , T ) = 0 means that no more interleaving instances can be located. If this condition holds, the whole test process terminates because this is the saturation point [11], [28] for IDAT-I, as for Maple, where additional test inputs are unlikely to locate any unknown interleaving instances.…”

Section: Definition 2 (Optimality) For a Candidate Set C Of Test Inpmentioning

confidence: 92%

See 1 more Smart Citation

Input-Driven Active Testing of Multi-threaded Programs

Han

Chen

et al. 2015

2015 Asia-Pacific Software Engineering Conference (APSEC)

View full text Add to dashboard Cite

It is still a challenge to select "good" test inputs for concurrent programs within limited testing resources. We present in this paper a test case diversity metric for multi-threaded programs, which evaluates a test input with its effect in exposing concurrent thread interactions. We then propose an input-driven active testing approach with two test input selection strategies based on our test case diversity metric. We implement our testing approach based on Maple, an interleaving coverage-driven active testing tool. The effectiveness and efficiency of our testing approach are compared closely with Maple, which on its own is supplied with random test inputs. Experimental results show that our testing approach can outperform the original active testing approach in the number of test inputs executed and the time usage for fulfilling the interleaving coverage criterion of Maple. The selected test inputs based on our test case diversity metric are very cost-effective in exposing concurrent thread interactions and hence can help detect concurrency bugs with less cost and effort.

show abstract

“…Using the IBM's concurrency testing infrastructure named ConTest, Bohuslav Krena et al [149] described a SBT environment for testing the concurrent programs. Further, coverage metric based on the algorithms like eraser, goldilocks, avio, goodlock, and happen-before have been proposed for the SBT of concurrent programs [150]. Figure 4 provides a citation record for the studies related to different approaches for testing a concurrent program.…”

Section: Model-based Testingmentioning

confidence: 99%

A systematic review of approaches for testing concurrent programs

Arora

Bhatia

Singh

2015

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYConcurrent programs are replacing the sequential programs as they utilize the true capabilities of multicore architecture. The extensive use of multicore systems and multithreaded paradigms warrants more attention to the testing of the concurrent programs. The testing concurrent program is not a new field as it has been more than 40 years because the first problem related to the testing concurrent program was addressed by the researchers. The field covers various domains, which include concurrency problems, testing approaches, techniques, graphical representations, tools, and subject systems. This paper aims at providing an overview of research in the domain of testing concurrent programs by classifying it into eight categories: (a) reachability testing, (b) structural testing, (c) model-based testing, (d) mutation-based testing, (e) slicing-based testing, (f) formal methods, (g) random testing, and (h) search-based testing. The survey is focused on the techniques applied, methodologies followed, and tools used in these aforementioned approaches. Furthermore, the gaps are also identified in different approaches. The paper concludes with the consolidation of various testing parameters along with the future directions.

show abstract

Coverage Metrics for Saturation-Based and Search-Based Testing of Concurrent Software

Cited by 18 publications

References 18 publications

Bita: Coverage-guided, automatic testing of actor programs

Bita: Coverage-guided, automatic testing of actor programs

Input-Driven Active Testing of Multi-threaded Programs

A systematic review of approaches for testing concurrent programs

Contact Info

Product

Resources

About