Traceability in Acceptance Testing

Abstract

Regardless of which (model-centric or code-centric) development process is adopted, industrial software production ultimately and necessarily requires the delivery of an executable implementation. It is generally accepted that the quality of such an implementation is of utmost importance. Yet current verification techniques, including software testing, remain problematic. In this paper, we focus on acceptance testing, that is, on the validation of the actual behavior of the implementation under test against the requirements of stakeholder(s). This task must be as objective and automated as possible. Our first goal is to review existing code-based and model-based tools for testing in light of what such an objective and automated approach to acceptance testing entails. Our contention is that the difficulties we identify originate mainly in a lack of traceability between a testable model of the requirements of the stakeholder(s) and the test cases used to validate these requirements. We then investigate whether such traceability is addressed in other relevant specification-based approaches.

Mapping ACL to JavaMOP: A Feasibility Study

“…This is accomplished through either the execution or static analysis of the IUT and generally consumes 40 -50 percent of development efforts and 40 -50 percent of development budgets [2,3]. As important as testing is, software is often not tested as thoroughly as it should be [4].…”

“…Orthogonal to these levels of testing, testing techniques can be categorized as either code-or model-centric [4].…”

“…Code-centric testing, such as Test-Driven Design (TDD), uses test cases written at the implementation level in order to guide development of the software [10]. These test cases do not rely on any form of requirements model and as a result provide no traceability to such a requirements model [4]. As well, the test cases are code artifacts and as such are implementation-driven and implementation-specific [4].…”

“…These test cases do not rely on any form of requirements model and as a result provide no traceability to such a requirements model [4]. As well, the test cases are code artifacts and as such are implementation-driven and implementation-specific [4]. This significantly reduces the reusability of the test cases.…”

“…The use of such an implementation-independent model provides two significant advantages: 1) A decoupling of requirements from the IUT and 2) The reuse of the test suite across several IUTs. [4] Of the various model-based approaches to system level testing, those built upon state-based specifications dominate [4,11]. These methods however have limitations that need to be overcome if a software testing tool is to scale up to industrial usage [4,12].…”