State Based Robustness Testing for Components

Lei, Bin; Liu, Zhiming; Morisset, Charles; Li, Xuandong

doi:10.1016/j.entcs.2009.12.037

Cited by 19 publications

(13 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The tool is applied to a benchmark of real-world components and shows its advantages compared to traditional approaches. This paper has been substantially revised and extended from the paper published in the proceedings of the FACS'08 workshop [21]. It is now firmly based on the formal theory of component and object software, rCOS.…”

Section: Discussionmentioning

confidence: 98%

Robustness testing for software components

Lei¹,

Liu³

et al. 2010

Science of Computer Programming

Self Cite

View full text Add to dashboard Cite

a b s t r a c tComponent-based development allows one to build software from existing components and promises to improve software reuse and reduce costs. For critical applications, the user of a component must ensure that it fits the requirements of the application. To achieve this, testing is a well-suited means when the source code of the components is not available. Robustness testing is a testing methodology to detect the vulnerabilities of a component under unexpected inputs or in a stressful environment. As components may fail differently in different states, we use a state machine based approach to robustness testing. First, a set of paths is generated to cover transitions of the state machine, and it is used by the test cases to bring the component into a specific control state. Second, method calls with invalid inputs are fed to the component in different states to test the robustness. By traversing the paths, the test cases cover more states and transitions compared to stateless API testing. We apply our approach to several components, including open source software, and compare our results with existing approaches.

show abstract

Section: Discussionmentioning

confidence: 98%

Robustness testing for software components

Lei¹,

Liu³

et al. 2010

Science of Computer Programming

Self Cite

View full text Add to dashboard Cite

show abstract

“…Dependencies of call orders as well as transition guards are not tested by JCrasher. Regarding the second category, the most recent approach is presented by Lei et al [13,12] in terms of the Robut Eclipse plugin. They present a methodology to check the behavior of state-based Java software components, which includes testing of defined and undefined transitions as well as postconditions of method calls using hostile parameters.…”

Section: Related Workmentioning

confidence: 99%

Accelerated robustness testing of state-based components using reverse execution

Heckeler

Schlich²,

Kropf

2013

Proceedings of the 28th Annual ACM Symposium on Applied Computing

View full text Add to dashboard Cite

This paper presents a methodology to test the robustness of reactive state-based software components by executing defined and undefined transitions of a UML Statechart specification, and therefore simulating a usage scenario under normal and abnormal conditions. The Z3 constraint solver is used to generate corresponding test cases. Since robustness testing is time-intensive due to a large number of undefined transitions to be tested, a main advantage of our presented approach is that it accelerates robustness testing by returning to reached states of a component, using reverse execution, and then refining them as a starting point for further tests. Furthermore, the presented approach provides runtime verification capabilities to test transition guards defined in the specification and therefore increases the test significance. This is done without instrumenting the source code. Due to the high acceleration achieved, our approach is well-suited for daily regression testing during component development.

show abstract

“…Also robustness testing is not taken into account. When discussing the state of the art regarding robustness testing of the structure of software components, the most resent approach is presented by Lei et al [30,31] in terms of the Robut Eclipse plugin. They present a methodology to check the behavior of statebased Java software components, which includes testing of defined and undefined transitions as well as post-conditions of method calls using hostile parameters.…”

Section: Related Workmentioning

confidence: 99%

Accelerated model-based robustness testing of state machine implementations

Heckeler

Eichelberger

Kropf

et al. 2013

SIGAPP Appl. Comput. Rev.

View full text Add to dashboard Cite

In this paper, we present an approach for accelerating model-based robustness testing of state-based software components. We use the Z3 SAT solver to derive executable test cases fully automatically from a state model based on a UML Statechart. The test cases are paths along the Statechart comprising states and transitions. The main advantage of our approach is an accelerated execution of the test cases by refining already reached states as starting points for further test cases using reverse execution. Furthermore, the presented approach is able to check whether a correct target state has been reached or not when executing the path-based test cases using runtime verification and therefore increases the test significance. The runtime verification is done without instrumenting the source code of the component under test 1Compared to common testing processes, robustness testing addresses all three mentioned types of errors (errors occurred by expected but not tested values, data related errors and errors occurred by misusing the interface of the component). To achieve this, robustness testing needs an increased number of test cases. Therefore, automatic test case generation is often used to create the test cases. But most test case generation approaches only focus on proper parameter data generation. Resulting test cases are then used to check whether an operation of a component returns the expected results regarding the input data when calling it. This kind of black-box testing approach is sufficient for state-less components. Such a component, also called transformation system, can for example implement a discrete cosine transformation. Every call to a parameterized operation receives input data, processes it, and returns the corresponding result. Further calls of operations work without remembering the previous calls. In contrast to stateless systems, state-based systems always have to be aware of

show abstract

State Based Robustness Testing for Components

Cited by 19 publications

References 8 publications

Robustness testing for software components

Robustness testing for software components

Accelerated robustness testing of state-based components using reverse execution

Accelerated model-based robustness testing of state machine implementations

Contact Info

Product

Resources

About