List of Tables
Executive SummaryThe work presented in this report was performed by the Carnegie Mellon® Software Engineering Institute (SEI) for the Army Strategic Software Improvement Program (ASSIP) and sponsored by the Army Program Executive Office (PEO) Aviation. This report presents a Virtual Upgrade Validation (VUV) approach to improving design quality and confidence in qualification through testing for military systems impacted by computer platform upgrades. This approach uses architecture-centric, model-based analysis to identify system-level problems early in the upgrade process to complement established test qualification techniques. For purposes of this report, the authors focus on changes to the computer platform consisting of processor, network, operating system, or runtime infrastructure.Helicopters and airplanes in military use today are operational well beyond their original life spans and typically are facing multiple platform upgrades as part of technology refresh cycles. Changes to the computer platform tend to be particularly risky because the embedded software makes many assumptions about the computer system. For example, software may have been developed for a federated architecture in which each software component is assumed to run on a dedicated, special processor using a cyclic executive as its runtime executive. The static nature of the task execution order may not be guaranteed on other computer platforms, affecting the execution order and timing. The emergence of the Integrated Modular Avionics (IMA) architecture provides the benefit of increased flexibility for growth of mission capability by utilizing a distributed computer system as a common computing platform. However, migration to this computer resource can have side effects not anticipated by the original embedded software application. For example, applications originally scheduled using a cyclic executive may now execute based on preemptive scheduling paradigms. As a result, the various control systems in the aircraft may encounter latency jitter and race conditions, due to nondeterministic sampling, that are difficult to detect through testing techniques. In one such case, the pilot experienced random blurring of the tracking symbol on his display screen due to latency jitter, which was traceable to nondeterministic sampling under certain processor load conditions [Feiler 1998]. This example illustrates that even planned upgrades to well-known standards-based architectures, such as Aeronautical Radio Incorporated (ARINC)653, can have impactful, unintended side effects.The U.S. Army has traditionally qualified systems and components by similarity, analysis, test, demonstration, or examination. Furthermore, current test approaches to achieving confidence in systems' airworthiness for the U.S. Army are based on traditional federated avionics systems [Boydston 2009]. The most common approach to dealing with platform change today is to port the code to the new platform and regression test exhaustively. Testers hope that the regressi...