Automatic generation of load tests

Zhang, Pingyu; Elbaum, Sebastian; Dwyer, Matthew B.

doi:10.1109/ase.2011.6100093

Cited by 60 publications

(44 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A Markov Model can be used to generate load test suites automatically [19,20,21,22,23]. Zhang presents a mixed symbolic execution approach aimed at discovering execution paths that contribute to high program loads while ensuring path diversity [24]. Malk presents a methodology to help automatically identify important performance counters for load testing and compare the counters across tests to find performance gain/loss [25].…”

Section: Detection Resultsmentioning

confidence: 99%

A Lightweight, Static Approach to Detecting Unbounded Thread-Instantiation Loops

Lei

Carver

et al. 2015

2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)

View full text Add to dashboard Cite

In server applications, threads are created to handle incoming requests. Since threads consume significant resources including CPU cycles and memory, it is important to control the number of threads that are created. In this paper, we introduce a lightweight, static approach to detecting unbounded threadinstantiation loops that may exist in a server application. The key observation of our approach is that threads are objects of special significance and the decision logic for thread instantiation is typically not complex. Our approach checks loops against some bounded thread-instantiation patterns. A loop is considered bounded if a pattern match is found. Otherwise, it is considered unbounded. Our approach is heuristic by nature. That is, it does not guarantee to detect all the unbounded loops and may report unbounded loops that are actually bounded. To evaluate the effectiveness of our approach, we report an Eclipse plugin called ThreadBoundChecker which implements our approach and an experiment on 24 real-life Java server applications. The results of our evaluation show that our approach can effectively detect unbounded thread-instantiation loops in these applications. In particular, 12 unbounded thread-instantiation loops detected by our approach are confirmed by the original developers.

show abstract

Section: Detection Resultsmentioning

confidence: 99%

A Lightweight, Static Approach to Detecting Unbounded Thread-Instantiation Loops

Lei

Carver

et al. 2015

2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST)

View full text Add to dashboard Cite

show abstract

“…This way, a bug in the code that processes option -o will be more likely to be exposed with DASE. On the other hand, the probability of hitting a bug in a shallower option (e.g., b) would be reduced from 15 , but the difference is much smaller, and it is still highly likely that the option b will be tested given that the probability is 15 . In addition to finding more bugs, since each option has about 1 15 chance to be explored, more options are likely to be tested, improving testing coverage ( § VI-B shows that DASE covers more options than KLEE).…”

Section: Overviewmentioning

confidence: 99%

“…Symbolic execution [7], [8] has been leveraged to automatically generate high code coverage test suites to detect bugs [9]- [15]. Symbolic execution represents inputs as symbolic values instead of concrete values.…”

Section: Introductionmentioning

confidence: 99%

DASE: Document-Assisted Symbolic Execution for Improving Automated Software Testing

Wong

Zhang

Wang

et al. 2015

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering

View full text Add to dashboard Cite

Abstract-We propose and implement a new approach, Document-Assisted Symbolic Execution (DASE), to improve automated test generation and bug detection. DASE leverages natural language processing techniques and heuristics to analyze program documentation to extract input constraints automatically. DASE then uses the input constraints to guide symbolic execution to focus on inputs that are semantically more important.We evaluated DASE on 88 programs from 5 mature real-world software suites: COREUTILS, FINDUTILS, GREP, BINUTILS, and ELFTOOLCHAIN. DASE detected 12 previously unknown bugs that symbolic execution without input constraints failed to detect, 6 of which have already been confirmed by the developers. In addition, DASE increases line coverage, branch coverage, and call coverage by 14.2-120.3%, 2.3-167.7%, and 16.9-135.2% respectively, which are 6.0-21.1 percentage points (pp), 1.6-18.9 pp, and 2.8-20.1 pp increases. The accuracies of input constraint extraction are 97.8-100%.

show abstract

“…In contrast to these existing approaches, SpeedGun addresses the challenges of generating concurrent performance tests. Zhang et al propose mixed symbolic-concrete test generation that focuses on paths that are estimated to have a high performance impact [59]. Wise generates tests that expose the worst-case complexity of a program [8].…”

Section: Test Generationmentioning

confidence: 99%

Performance regression testing of concurrent classes

Pradel

Huggler

Groß

2014

Proceedings of the 2014 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Developers of thread-safe classes struggle with two opposing goals. The class must be correct, which requires synchronizing concurrent accesses, and the class should provide reasonable performance, which is difficult to realize in the presence of unnecessary synchronization. Validating the performance of a thread-safe class is challenging because it requires diverse workloads that use the class, because existing performance analysis techniques focus on individual bottleneck methods, and because reliably measuring the performance of concurrent executions is difficult. This paper presents SpeedGun, an automatic performance regression testing technique for thread-safe classes. The key idea is to generate multi-threaded performance tests and to compare two versions of a class with each other. The analysis notifies developers when changing a thread-safe class significantly influences the performance of clients of this class. An evaluation with 113 pairs of classes from popular Java projects shows that the analysis effectively identifies 13 performance differences, including performance regressions that the respective developers were not aware of.

show abstract

Automatic generation of load tests

Cited by 60 publications

References 15 publications

A Lightweight, Static Approach to Detecting Unbounded Thread-Instantiation Loops

A Lightweight, Static Approach to Detecting Unbounded Thread-Instantiation Loops

DASE: Document-Assisted Symbolic Execution for Improving Automated Software Testing

Performance regression testing of concurrent classes

Contact Info

Product

Resources

About