Automated Link-16 Testing Using the Discrete Event System Specification and Extensible Markup Language

Abstract

With the modernization of Department of Defense (DoD) systems and the growing complexity of communication equipment, traditional test methods and processes have to evolve in order to maintain their effectiveness. DoD acquisition policy requires the use of modeling and simulation (M&S) in all phases of system development life-cycles in order to ensure technical certification and mission effectiveness. The complexity of these systems poses significant challenges over traditional interoperability test methodologies. The Automated Test Case Generator (ATC-Gen), funded by the Joint Interoperability Test Command (JITC), captures a Military Standard (MIL-STD) 6016C document and translates it into rules. These are in turn formalized into test cases using the Discrete Event System Specification (DEVS). In this paper, we present a new methodology to generate the test models and perform conformance testing using system theory, the DEVS M&S framework, the System Entity Structure (SES), and Extensible Markup Language (XML). This new methodology promotes the separation of the models, the simulator, and the distributed simulation. These separations distinguish and promote reusability by developing models, the simulator, and distributed simulation independently. The DEVS test models are generated from the test cases by the Test Model Generator using the system specifications. These models are written in an XML-SES format; the resulting C++ DEVS source code is generated based on the test model XML file. The Test Driver (TD) was designed based on the Model/Simulator/View/Control (MSVC) design pattern and developed to execute the DEVS test models. MSVC supports model and simulator separation design. It was also designed to support multiple network simulation protocols and rapid software modifications in order to incorporate new network protocols into the simulation software. This methodology was used to verify the conformance of the Integrated Architecture Behavior Model (IABM) to the MIL-STD 6016C, and the results of the test scenarios were validated using the JITC's Simple J network packet-monitoring tool. The network packet monitor captured the transmissions and the receipt of the tactical data messages from the TD. The system analyst interpreted and verified the messages, and determined whether these messages were the intended behavior of the TD.

Model-based tool support for Tactical Data Links: an experience report from the defence domain

“…The decomposition results in two types of specifications: the hierarchical structure of the system modules and the internal structure, or the state transition, of the system modules. Mak et al [29] discuss an automated testing framework based on DEVS modelling and simulation formalism, XML and System Entity Structure (SES). This framework is part of the Automated Test Case Generator (ATC-Gen) funded by the United States Department of Defense Joint Interoperability Test Command (JITC) to support the mission of standards compliance and certification.…”

“…Finally, tests are generated as DEVS C++ test models, these are executed against the System Under Test (SUT) using a Test Driver. The Test Driver performs SUT conformance testing by inducing the testable behaviour expressed in the models into SUT and checking the responses for accuracy [29]. This work is interesting because it seeks to formalise the Link 16 standard such that it is processable by a machine.…”

Model-based testing methodology using system entity structures for MATLAB/Simulink models

“…Analogous to Mak et al 16 we are referring to the simulation theory to fulfil these preconditions. Since the testing of simulation models can be seen as experimenting with a simulation model, we employ, as in Mak et al, 16 the concept of an experimental frame (EF) for formally specifying simulation test models. The EF concept originates from the discrete event system specification (DEVS).…”

“…In combination with an MB, distinct simulation models can be derived from an SES. 18 In Mak et al 16 and Schmidt et al 19 it is already assumed to specify different test model designs using an SES. Hence, an SES describes a set of various MUT and test model structures and their parameter settings.…”

“…Then, based on the SES and MB, a specific executable model of an MUT and a test model can be generated. The methodology in this paper, for specifying test model designs on an abstract level and for deriving executeable models from it, is similar to the ideas presented by Mak et al 16 and Schmidt et al 19 However, some details are quite different, especially with regard to the design of the test models, MUTs and the objectives for the infrastructure prototyped for MATLAB/Simulink. MATLAB/Simulink from the MathWorks, Inc. 20 is the most popular model-based design and simulation environment for CPS.…”