A Cybernetic Analysis of Biodynamic Effects in Touchscreen Operation in Turbulence

Abstract

This paper describes a human-in-the-loop experiment performed in TU Delft's SIMONA Research Simulator to explicitly investigate the effects of biodynamic feedthrough (BDFT) on the execution of a two-dimensional touchscreen waypoint dragging task in turbulence. In the experiment, 16 participants performed the same task in a stationary simulator and whilst being perturbed in either surge, sway, or heave directions by the same motion disturbance signal. In addition, the effect of screen location on biodynamic effects was tested by considering two touchscreen display positions, i.e., representative for the Primary Flight Display (PFD) and the Control Display Unit (CDU), respectively. The collected results show significantly more issues with loss of screen contact for the PFD display, due to the extended arm position when operating this display. Despite the fact that small biodynamic effects are also found for off-axis disturbances, the results clearly show that issues with BDFT predominantly occur when motion disturbances are aligned with the touchscreen control input direction. Due to the use of a multisine motion disturbance signal in the experiment, explicit BDFT detection and identification was possible with spectral methods. For the conditions where sufficient BDFT was detected to allow for modeling the BDFT dynamics, a second-order BDFT model was proposed and found to explain at least 70% of the BDFT input components. With consistent BDFT model parameter estimates between conditions, this approach shows clear merit for quantitative analysis of biodynamic effects on touchscreen operation and potentially even for BDFT mitigation through model-based input cancellation. Nomenclature

Mitigation of Biodynamic Feedthrough for Touchscreens on the Flight Deck

“…For the multisine task, the horizontal (f t y ) and vertical (f t z ) target screen positions were defined as sum-of-sine signals, resulting in a uninterrupted required movement across the screen. These signals were identical to those used by Mobertz et al (2018) and consisted of three sines with distinct frequencies. The two-dimensional touchscreen target signals were meant to create a continuous and unpredictable task for the operator, without being too challenging.…”

“…Figure 3 shows a sample time trace for the horizontal target signal f t y , as well as the corresponding recorded touchscreen input u y . For reference, Table 1 lists all details of the f t y and f t z signals as also reported in (Mobertz et al, 2018).…”

“…In our experiment, the motion disturbance signal f d y;z (see Figure 1) was used to simulate motion accelerations representative for realistic turbulence, while at the same time enabling the retrieval of an estimate of participants' BDFT dynamics (H BDFT ) using frequency-domain system identification techniques (Damveld et al, 2013(Damveld et al, , 2010Mobertz et al, 2018;Van Paassen & Mulder, 2006), see Sec. 3.3.…”

“…One key reason for decreased task performance is biodynamic feedthrough (BDFT): the involuntary movement of limbs due to physical accelerations or vibrations (Mobertz et al, 2018;Venrooij, 2014;Venrooij et al, 2013). BDFT causes parts of the body to move in an unintentional manner, which is known to result in involuntary and undesired direct feedthrough of perturbed arm/hand movements into touchscreen gesture inputs, as often happens when operating a smart phone while walking.…”

“…Although BDFT so far has received little attention in regard to touchscreen operation, it has been extensively investigated for other settings where it causes problems, such as input and control tasks with traditional input devices in aircraft and helicopters (Allen et al, 1973;Jex, 1972;Masarati et al, 2015;Mayo, 1989;Venrooij, 2014), hydraulic excavators (Humphreys et al, 2010), and electric wheelchairs (Banerjee et al, 1996). Only a single previous study (Mobertz et al, 2018) has focused on the explicit quantification of BDFT when using a touchscreen in a moving environment. For the effective use of touchscreens on the modern flight deck in all flight conditions, it is essential that an effective approach to the mitigation of BDFT, which can minimize the occurrence of possibly hazardous touchscreen input errors, is developed.…”