A Simulator Comparison Study into the Effects of Motion Filter Order on Pilot Control Behavior

Abstract

This paper describes an experiment investigating the effects of motion filter order on human manual control tracking behavior and performance. The experiment was performed on two simulators: the Vertical Motion Simulator at NASA Ames Research Center and the SIMONA Research Simulator at Delft University of Technology. Eighteen pilots in the Vertical Motion Simulator and twenty pilots in the SIMONA Research Simulator performed the experiment with a full factorial variation of three motion filter orders and two motion filter frequencies, in addition to a reference no-motion and full-motion condition. Motion shaping filters derived from Objective Motion Cueing Test measurements on the Vertical Motion Simulator were included in the SIMONA Research Simulator motion logic to match the motion cues between both simulators. Furthermore, the side sticks were set to matching characteristics and the visual cues were matched in terms of time delay, graphics size and screen characteristics. With increased motion filter order, pilots showed worse performance and a lowered contribution of motion feedback in their control strategy. Increasing the motion filter break frequency had similar effects, which were stronger than the effects of increasing the motion filter order, for the eight experimental conditions that were considered in this experiment. For the same motion condition the simulators showed offsets in the results. However, the trends between the motion conditions were similar, leading to the conclusion that for simulator comparisons relative trends are easier to replicate between simulators than absolute results within one condition.

Simulator Assessment of the Lateral-Directional Handling Qualities of the Flying-V

“…The SRS is a high fidelity, six-degree of freedom ground-based research simulator, initially developed for advanced research into simulation techniques, motion system control and navigation systems technologies [37]. Ever since, it has been used for a broad range of research such as piloted handling quality assessments [38] and pilot control behavior research [39]. The SRS was shown to have correlated results to simulators leading in fidelity such as the Calspan Total In-Flight Simulator (TIFS) and the NASA Ames Vertical Motion Simulator (VMS) [40].…”

Linear Mixed-Effects Models for Human-in-the-Loop Tracking Experiment Data

“…For Experiment 1, which did not have a factorial set of test conditions defined by multiple independent variables, see Table 1, a 1-way repeated measures ANOVA was used as the main statistical test [1]. In cases where data distributions were strongly non-normal (e.g., 2 ), a nonparametric Friedman test was applied. Experiment 2 tested a complete factorial combination of three and three settings, in addition to a reference no-motion condition S(0.0/-).…”

“…The original statistical analysis for Experiment 2 [9,14] was performed using a 2-way factorial repeated measures ANOVA on all conditions with simulator motion, with and as statistical factors, hence excluding the no-motion case. Overall, the statistical comparisons of the data reported in [1,9,14] were suboptimal, due to problems with data distributions (e.g., 2 in Experiment 1) and the fact that the tested conditions were not a balanced factorial combination of independent variables (Experiments 1 and 2).…”

“…In addition, due to the fact that it is often not possible (e.g., to avoid experiments becoming impractically long in duration) to test a complete factorial set of experiment conditions, in many cases researchers compromise for a statistically incomplete design. Finally, key dependent measuresthat are often used for comparing across different conditions (e.g., performance metrics, crossover frequency) typically do not provide well-distributed (i.e., normal) samples, due to large between-subject differences or sub-populations in the subject pool (e.g., high/low-gain pilots [1][2][3]). Despite these typical issues, the default choice for statistical analysis in most human-in-the-loop simulation studies is still to perform a (repeated-measures) analysis of variance (ANOVA), or a robust alternative [1,2,[4][5][6][7][8][9][10][11].…”