Manual Versus Speech Input for Unmanned Aerial Vehicle Control Station Operations

Abstract

Unmanned aerial vehicle (UAV) control stations feature multiple menu pages with systems accessed by keyboard presses. Use of speech-based input may enable operators to navigate through menus and select options more quickly. This experiment examined the utility of conventional manual input versus speech input for tasks performed by operators of a UAV control station simulator at two levels of mission difficulty. Pilots performed a continuous flight/navigation control task while completing eight different data entry task types with each input modality. Results showed that speech input was significantly better than manual input in terms of task completion time, task accuracy, flight/navigation measures, and pilot ratings. Across tasks, data entry time was reduced by approximately 40% with speech input. Additional research is warranted to confirm that this head-up, hands-free control is still beneficial in operational UAV control station auditory environments and does not conflict with intercom operations and intra-crew communications.

NtoM: a Concept of Operations for Pilots of Multiple Remotely Piloted Aircraft

“…Therefore, in this respect the interface should address a couple of issues; first, trying to reduce the involvement of cognitive resources in the loading of the second flight situation. Maybe the possibility to reply to background messages using voice recognition (partially implemented but not used during the experiments) to avoid the overload of visual resources, could alleviate some of these situations, a task interference solution as shown in [49]- [51] and with a promising performance in [52]. Second, despite their natural inclinations, tasks with higher priority should not be deferred in favour of those with lower priority, whatever the mental workload each of them involves.…”

Kite: Automatic Speech Recognition for Unmanned Aerial Vehicles

“…Speech recognition for UAV control. The task of speech recognition for UAV control is relatively unexplored and the few published works on this topic [4,5,10] focus on recognition of simple commands: the authors of [4] predict a fixed set of nine commands using a classification pipeline based on audio features, such as energy and MFCC; the method in [10] recognizes commands to navigate through menus, operations which were previously achieved through keyboard presses.…”

Effects of Touch, Voice, and Multimodal Input, and Task Load on Multiple-UAV Monitoring Performance During Simulated Manned-Unmanned Teaming in a Military Helicopter

“…Few studies have systematically evaluated how the choice of interface input method affects operator performance with UAV control systems (see Calhoun, Ruff, Behymer, & Rothwell, 2017;Draper, Calhoun, Ruff, Williamson, & Barry, 2003;Taylor et al, 2015). None have evaluated systems designed for an AMC located in a Black Hawk helicopter, which is important because it is critical to consider the specific application when designing interfaces for multi-UAV control (Calhoun & Draper, 2015).…”

“…First, environmental factors such as vibration, glare, and cockpit noise could reduce the effectiveness of touchscreen displays and voice recognition systems (Cockburn et al, 2017;Noyes & Haas, 2010), and these were not simulated in our study. Additionally, our system did not use features that make voice control especially useful, such as the consolidation of multiple touch inputs into a single "voice macro" (Draper et al, 2003). Finally, because our evaluation involved the comparison of a particular touchscreen display to a particular voice recognition system/vocabulary, our results do not necessarily generalize to all such implementations of these technologies (see Supplement A for additional discussion of this topic).…”