Development of an integrated simulation platform for real-time task performance assessment

Abstract

A re-configurable, portable test station was developed for integrating and testing real-time performance metrics for continuously assessing operator effectiveness in operationally-relevant spaceflight piloting tasks. The test station includes a single computer for hosting the vehicle simulation, rendering both graphical flight displays and a 3-D out-the-window view, and computing the performance metrics in real-time. The pilot interacts with the simulation using four displays (two piloting displays, one out-the-window display, and a mission summary display), a rotational hand controller, a translational hand controller, and a microphone. A fifth display provides a system status I engineering view for the experimenter. A key component of the simulation station is the real-time metrics engine and algorithms, which estimates pilot workload, situation awareness, and flight performance without interfering with the piloting task, or adding equipment or infrastructure to the flight deck. Workload and flight performance are estimated based on an analysis of the vehicle state (e.g., attitude, altitude, % fuel) and the pilot commands (e.g., hand controller movement), whereas situation awareness is estimated based on the comparison of the actual vehicle state and that spoken (and converted to text through an automatic speech recognition algorithm) by the flying pilot. This real time simulation station development is discussed in the context of four operationally-relevant spaceflight tasks: piloted lunar landing, Orion/MPCV docking operations with the International Space Station (ISS), and manual control of the spacewalking Simplified Aid for EVA Rescue (SAFER) jet pack near the ISS.

Longitudinal Impacts of Simulated Long-Duration Spaceflight Missions on Operationally Relevant Measures of Human Performance Using a Portable Simulation Platform

“…Non-parametric analyses were selected given the non-normal distribution of the primary data variables of interest. Use of the non-parametric tests is consistent with past data analyses from previous laboratory-based experiments (Duda et al, 2015). To assess the effects of mission duration, the data for each metric were analyzed by average values across mission quartiles using Friedman tests.…”

“…The objective of this work was to investigate the longitudinal impacts of simulated, long-duration space exploration missions on operationally relevant measures of human performance. Based on prior work showing mood impacts of long-duration missions (e.g., Bechtel & Berning, 1991; Steel, 2001; and Van Wijk, 2018), we hypothesized that time in simulation would indeed effect performance measures as assessed by the Draper-developed RTPM (described above, and in Duda et al, 2015).…”

“…These hand controllers can be reconfigured to control a wide variety of vehicle dynamics, allowing for additional simulation flexibility. A commercial, off-the-shelf, noise-canceling, head-mounted microphone is used to unobtrusively collect spoken audio data from the operator to estimate situation awareness in real-time (while this function is outside the scope of the present investigation, more detail of this capability can be found in Duda et al, 2015).…”

“…Mental workload is typically measured via assessment of spare attentional capacity, either directly by employing subjective scales, such as the hierarchical "Modified Bedford Workload Scale" (Newman & Greeley, 2001;Oman et al, 1995;Roscoe & Ellis, 1990), self-report questionnaires, like the "NASA Task Load Index" (NASA-TLX) (Hart & Staveland, 1988), or by quantifying performance on a secondary task. Embedded secondary monitoring tasks that use the same sensory resources as the primary task (e.g., visual modality), and are well integrated to be part of the operator's normal role, are considered most effective and least obtrusive (Hainley, 2011;Hainley et al, 2013;Duda et al, 2015).…”

“…The development and architecture (both hardware and software) of the Draper RTPM (illustrated in Figure 1) has been previously described in detail (Duda et al, 2015). Briefly, this system is composed of a single multi-core, multi-graphics processing unit computer capable of executing the vehicle simulation, rendering both graphical flight displays and a three-dimensional out-the-window view, and computing the multi-dimensional performance metrics in real-time.…”