Software Tools to Support Task-Based Operational Test and Evaluation

Abstract

The Air Force Operational Test and Evaluation Center (AFOTEC) is implementing a new approach to the operational test and evaluation (OT&E) process. This approach, task-based OT&E, focuses on the tasks performed by users of a system under test. Operator tasks will be analyzed and used to evaluate C4I systems by “rolling up” task performance in a larger, mission-level framework. As a means of evaluating the system's operational effectiveness, operational testers will assess the system's contribution to overall mission accomplishment. The Test and Evaluation Research and Education Center (TEREC) at the Georgia Institute of Technology was contracted by AFOTEC to characterize and evaluate the task-based OT&E process and then propose a set of commercially available software tools to support the operational tester. This paper describes the task-based OT&E process and the results of a tool requirements analysis. Based on requirements derived from this analysis, a set of commercially available software tools was recommended. From a human factors perspective, task-based OT&E presents an interesting challenge. Specifically, it requires human factors support at two user levels: the system operator level and the operational tester level.