Automated, contract-based user testing of commercial-off-the-shelf components

Abstract

Commercial-off-the-Shelf (COTS) components provide a means to construct software (component-based) systems in reduced time and cost. In a COTS component software market there exist component vendors (original developers of the component) and component users (developers of the component-based systems).The former provide the component to the user without source code or design documentation, and as a result it is difficult for the latter to adequately test the component when deployed in their system. In this article we propose a framework that clarifies the roles and responsibilities of both parties so that the user can adequately test the component in a deployment environment and the vendor does not need to release proprietary details. Then, based on this framework we combine and adapt two specificationbased testing techniques and describe (and implement) a method for the automated generation of adequate test sets. An evaluation of our approach on a case study demonstrates that it is possible to automatically generate cost effective test sequences and that these test sequences are effective at detecting complex errors.

The use of mutation in testing experiments and its sensitivity to external threats

“…Briand et al [6] evaluated their framework for testing commercial space printtokens printtokens2 replace schedule schedule2 tcas totinfo NLOC 5,905 343 355 513 296 263 137 281 Test Cases 13,585 4,130 4,115 5,542 2,650 2,710 1,608 1,052 Faults 38 7 10 32 9 10 41 23 Non-Eq. Faults 34 5 10 31 9 9 41 23 Mutants Generated 301,400 12,536 11,284 23,369 3,813 6,938 4,721 10,863 Mutants Selected (10%) 30,140 1,253 1,128 2,336 381 693 472 1,086 Mutants Compiled 26,263 1,103 988 2,336 381 620 470 843 Eq.…”

“…6 Some of MMRE values were not computable and were marked Inf as shown in Table 4. four Java classes from the Docjar Web site [1] as listed in Table 5.…”

Whitening SOA Testing via Event Exposure

“…The joint execution of both tasks represents an inconsistency fault. To filter equivalent mutants, 14 we test every faulty version using all the test cases in the test pool. 15 If a mutant cannot be detected by any test case in the test pool, we regard it as an equivalent mutant and remove it from the set of faulty versions.…”

“…Many research efforts have been devoted to testing component-based software [13], [49], [50], wherein the source code and design of a component may also be unavailable. Therefore, metadata of components such as constraints on succeeding and preceding events are provided for testing purposes [14], [62]. This approach is similar to ours, in the sense that component vendors need to provide customers with interfaces and probes to observe internal states of components for testing purposes.…”

Uso da técnica de teste estrutural para o teste e monitoração de serviços

“…Após fazer essa marcação, o relatório de cobertura será atualizado com a nova cobertura alcançada. Briand et al (2006) também apresentaram um trabalho relacionado ao uso de metadados para apoiar o teste de componentes. O desenvolvedor gera restrições CSPE (Constraints on Succeeding and Preceding Events) para as operações das interfaces do componente.…”