The Need for Unified Testbed Management Across Multiple Teams and Stakeholders in a Large Scale Telecom Integration Context

Abstract

The quality assurance of large scale integrative systems often requires complex testbed environments and simulations that allow to test the overall functionality and enables various experiments towards systematically verifying the realization of the identified user and system requirements. Thereby, an integration setup and resulting activities lead to another level of quality assurance, whereby the integrator deals with the quality examination of the single components and their integrative interplay according to a set of overall system and user requirements. In such context, it is often the case that the testing activity is conducted by various partners (e.g. single companies and legal entities) with complementing know how required for specific sub-tasks -e.g. PKI, chip cards, special network protocols, firewall, security architectures, and penetration testing. This leads to the emergence of a large number of proprietary testbeds focusing on specific aspects resulting in the lack of a unified testbed configuration, versioning and technological foundations (e.g. operating system, network stack implementations, hypervisor technology …). In this paper, we present our experiences drafted from a large scale industrial project with 600-700 requirements relating to a critical eHealth infrastructure within a telecom provider context. Thereby, various sub-contractors had to be unified in their approach to testbed management in order to achieve reproducible and traceable (with respect to system requirements) test results based on a test architecture accommodating various quality assurance activities (unit testing, development tests, component testing, integration testing, security testing …). We gradually analyze the project situation with respect to testbed management and argue on the need for unified testbed management across multiple teams and stakeholders in a large scale telecom integration setup. Subsequently, we propose possible solutions and conduct a series of experiments highlighting the advantages of the proposed approach and belonging solution.