Structural Specification-Based Testing: Automated Support and Experimental Evaluation

Chang, Juei; Richardson, Debra J.

doi:10.1007/3-540-48166-4_18

Cited by 34 publications

(43 citation statements)

References 26 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This is different from the problems that traditional test selection techniques address [6,21]. In those problems, there are test oracles for unselected test inputs.…”

Section: Unit-test Generation and Selectionmentioning

confidence: 94%

“…For fault detection, a new feature could be fault-revealing behavior that does not occur during the execution of the existing tests. In black-box test generation, a new feature could be a predicate in the specifications for the unit [6]. In white-box test generation, a new feature could be program behavior exhibited by executing a new structural entity, such as statement, branch, or def-use pair.…”

Section: Unit-test Generation and Selectionmentioning

confidence: 99%

“…In white-box test augmentation, we select a test that can decrease residual structural coverage and thus increase structural coverage. In black-box test augmentation, we select a test that covers a new predicate in a priori specifications [6].…”

Section: Unit-test Generation and Selectionmentioning

confidence: 99%

“…All these specification-based test generation tools require a priori specifications. When specifications are provided for a unit a priori, Chang and Richardson use specification coverage criteria to suggest a candidate set of test cases that exercise new aspects of the specification [6]. Our approach uses generated operational abstractions to guide test generation tools and select generated tests without requiring a priori specifications.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Tool-assisted unit-test generation and selection based on operational abstractions

Xie

Notkin

2006

Autom Software Eng

View full text Add to dashboard Cite

Abstract. Unit testing, a common step in software development, presents a challenge. When produced manually, unit test suites are often insufficient to identify defects. The main alternative is to use one of a variety of automatic unit-test generation tools: these are able to produce and execute a large number of test inputs that extensively exercise the unit under test. However, without a priori specifications, programmers need to manually verify the outputs of these test executions, which is generally impractical. To reduce this cost, unit-test selection techniques may be used to help select a subset of automatically generated test inputs. Then programmers can verify their outputs, equip them with test oracles, and put them into the existing test suite. In this paper, we present the operational violation approach for unit-test generation and selection, a black-box approach without requiring a priori specifications. The approach dynamically generates operational abstractions from executions of the existing unit test suite. These operational abstractions guide test generation tools to generate tests to violate them. The approach selects those generated tests violating operational abstractions for inspection. These selected tests exercise some new behavior that has not been exercised by the existing tests. We implemented this approach by integrating the use of Daikon (a dynamic invariant detection tool) and Parasoft Jtest (a commercial Java unit testing tool), and conducted several experiments to assess the approach.

show abstract

“…This is different from the problems that traditional test selection techniques address [6,21]. In those problems, there are test oracles for unselected test inputs.…”

Section: Unit-test Generation and Selectionmentioning

confidence: 94%

Section: Unit-test Generation and Selectionmentioning

confidence: 99%

Section: Unit-test Generation and Selectionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Tool-assisted unit-test generation and selection based on operational abstractions

Xie

Notkin

2006

Autom Software Eng

View full text Add to dashboard Cite

show abstract

“…Identified features in a program can be statements, branches, paths, or definitionuse paths. Identified features in a specification can be choices for categories [4,5] or conditions [12] in specifications.…”

Section: Related Workmentioning

confidence: 99%

Defining and Measuring Policy Coverage in Testing Access Control Policies

Martin

Xie

2006

Information and Communications Security

View full text Add to dashboard Cite

Abstract. To facilitate managing access control in a system, security officers increasingly write access control policies in specification languages such as XACML, and use a dedicated software component called a Policy Decision Point (PDP). To increase confidence on written policies, certain types of policy testing (often in an ad hoc way) are usually conducted, which probe the PDP with some typical requests and check PDP's responses against expected ones. This paper develops a first step toward systematic policy testing by defining and measuring policy coverage when testing policies. We have developed a coverage-measurement tool to measure policy coverage given a set of XACML policies and a set of requests. We have developed a tool for request generation, which randomly generates requests for a given set of policies, and a tool for request reduction, which greedily selects a nearly minimal set of requests for achieving the same coverage as the originally generated requests. To evaluate coverage-based request reduction and its effect on fault detection, we have conducted an experiment with mutation testing on a set of real policies. Our experimental results show that the coverage-based test reduction can substantially reduce the size of generated requests and incur only relatively low loss on fault detection. We also conduct a study on the policy coverage achieved by manually generated requests.

show abstract

Directed test suite augmentation: an empirical investigation

Kim

et al. 2014

Software Testing Verif & Rel

View full text Add to dashboard Cite

SummaryTest suite augmentation techniques are used in regression testing to identify code elements in a modified program that are not adequately tested and to generate test cases to cover those elements. A defining feature of test suite augmentation techniques is the potential for reusing existing regression test suites. Our preliminary work suggests that several factors influence the efficiency and effectiveness of augmentation techniques that perform such reuse. These include the order in which target code elements are considered while generating test cases, the manner in which existing regression test cases and newly generated test cases are used, and the algorithm used to generate test cases. In this work, we present the results of two empirical studies examining these factors, considering two test case generation algorithms (concolic and genetic). The results of our studies show that the primary factor affecting augmentation using these approaches is the test case generation algorithm utilized; this affects both cost and effectiveness. The manner in which existing and newly generated test cases are utilized also has a substantial effect on efficiency and in some cases a substantial effect on effectiveness. The order in which target code elements are considered turns out to have relatively few effects when using concolic test case generation but in some cases influences the efficiency of genetic test case generation. The results of our first study, on four relatively small programs using a large number of test suites, are supported by our second study of a much larger program available in multiple versions. Together, the studies reveal a potential opportunity for creating a more cost‐effective hybrid augmentation approach leveraging both concolic and genetic test case generation techniques, while appropriately utilizing our understanding of the factors that affect them. Copyright © 2014 John Wiley & Sons, Ltd.

show abstract

Structural Specification-Based Testing: Automated Support and Experimental Evaluation

Cited by 34 publications

References 26 publications

Tool-assisted unit-test generation and selection based on operational abstractions

Tool-assisted unit-test generation and selection based on operational abstractions

Defining and Measuring Policy Coverage in Testing Access Control Policies

Directed test suite augmentation: an empirical investigation

Contact Info

Product

Resources

About