An automatically-generated run-time instrumenter to reduce coverage testing overhead

Li, J. Jenny; Weiss, David M.; Yee, Howell

doi:10.1145/1370042.1370054

Cited by 8 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Agrawal [1] optimizes probes by forming "superblocks" from sets of basic blocks based on dominance and post-dominance relations. Later, Agrawal [2] extends this work to interprocedural dominance relations; Li et al [21] and Xu et al [39] extend these optimizations beyond superblocks. Tikir and Hollingsworth [37] also optimize coverage probe placement via dominators, but use a faster, simpler approach (and no post-dominance information) for online instrumentation.…”

Section: Related Workmentioning

confidence: 99%

Optimizing customized program coverage

Ohmann

Brown

Neelakandan

et al. 2016

Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

View full text Add to dashboard Cite

Program coverage is used across many stages of software development. While common during testing, program coverage has also found use outside the test lab, in production software. However, production software has stricter requirements on run-time overheads, and may limit possible program instrumentation. Thus, optimizing the placement of probes to gather program coverage is important. We introduce and study the problem of customized program coverage optimization. We generalize previous work that optimizes for complete coverage instrumentation with a system that adapts optimization to customizable program coverage requirements. Specifically, our system allows a user to specify desired coverage locations and to limit legal instrumentation locations. We prove that the problem of determining optimal coverage probes is NP-hard, and we present a solution based on mixed integer linear programming. Due to the computational complexity of the problem, we also provide two practical approximation approaches. We evaluate the effectiveness of our approximations across a diverse set of benchmarks, and show that our techniques can substantially reduce instrumentation while allowing the user immense freedom in defining coverage requirements. When naïve instrumentation is dense or expensive, our optimizations succeed in lowering execution time overheads. CCS Concepts •Software and its engineering → Software testing and debugging; Software post-development issues; Software performance; •Mathematics of computing → Integer programming;

show abstract

Section: Related Workmentioning

confidence: 99%

Optimizing customized program coverage

Ohmann

Brown

Neelakandan

et al. 2016

Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

View full text Add to dashboard Cite

show abstract

“…Most of the tools reviewed in [39] use source code instrumentation, while a smaller number use byte code instrumentation (including JCover and PurifyPlus) or dynamic (runtime) instrumentation [1]. Regardless of the instrumentation approach, all of the coverage tools described in [39] have a reported instrumentation overhead of more than 30% [17].…”

Section: Related Workmentioning

confidence: 99%

A Flexible and Non-intrusive Approach for Computing Complex Structural Coverage Metrics

Whalen

Person

Rungta

et al. 2015

2015 IEEE/ACM 37th IEEE International Conference on Software Engineering

View full text Add to dashboard Cite

Software analysis tools and techniques often leverage structural code coverage information to reason about the dynamic behavior of software. Existing techniques instrument the code with the required structural obligations and then monitor the execution of the compiled code to report coverage. Instrumentation based approaches often incur considerable runtime overhead for complex structural coverage metrics such as Modified Condition/Decision (MC/DC). Code instrumentation, in general, has to be approached with great care to ensure it does not modify the behavior of the original code. Furthermore, instrumented code cannot be used in conjunction with other analyses that reason about the structure and semantics of the code under test. In this work, we introduce a non-intrusive preprocessing approach for computing structural coverage information. It uses a static partial evaluation of the decisions in the source code and a source-to-bytecode mapping to generate the information necessary to efficiently track structural coverage metrics during execution. Our technique is flexible; the results of the preprocessing can be used by a variety of coverage-driven software analysis tasks, including automated analyses that are not possible for instrumented code. Experimental results in the context of symbolic execution show the efficiency and flexibility of our nonintrusive approach for computing code coverage information.

show abstract

“…Similarly to Tikir et al, Li et al [25] combine selective instrumentation and removal of no longer needed instrumentation to decrease the overhead of code coverage instrumentation. They propose to do an offline analysis step with their super nested block algorithm, which aims at reducing the number of required probes and is similar to the dominator-based approach of Agrawal.…”

Section: Disposable Instrumentationmentioning

confidence: 99%

Deriving code coverage information from profiling data recorded for a trace-based just-in-time compiler

Häubl

Wimmer

Mössenböck

2013

Proceedings of the 2013 International Conference on Principles and Practices of Programming on the Java Platform: Virtual Machi

View full text Add to dashboard Cite

Code coverage information is typically recorded by adding instrumentation to an application before or during execution. However, this has the disadvantage that the instrumentation decreases the performance.A high-performance virtual machine (VM), such as Oracle's Java HotSpot VM, is already designed to record profiling data for the executed application. This profiling data is intended to guide optimizations during just-in-time (JIT) compilation. In this paper, we propose a runtime system that allows deriving exact code coverage information from this recorded profiling data by guaranteeing certain system properties. With this approach, we avoid the hassle of instrumenting the application explicitly. Therefore, we minimize both the run-time overhead and the implementation effort for recording code coverage information.We implemented our approach for a variant of the Java HotSpot VM that uses a trace-based JIT compiler. However, our approach is simple and general enough to be applicable to most modern VMs. Our evaluation with industry-strength benchmark suites shows that this approach allows recording code coverage information while retaining almost full peak performance. So, our approach is also suitable for collecting code coverage information in day-to-day operation environments such as long running server applications.

show abstract

An automatically-generated run-time instrumenter to reduce coverage testing overhead

Cited by 8 publications

References 10 publications

Optimizing customized program coverage

Optimizing customized program coverage

A Flexible and Non-intrusive Approach for Computing Complex Structural Coverage Metrics

Deriving code coverage information from profiling data recorded for a trace-based just-in-time compiler

Contact Info

Product

Resources

About