Optical Brain Imaging to Enhance UAV Operator Training, Evaluation, and Interface Development

A meta-analysis of human-system interfaces in unmanned aerial vehicle (UAV) swarm management

“…For example, a swarm engaged in 'foraging' or searching for something over a defined area will need less direct human guidance but more human analysis in determining if the object has been found (Bashyal and Venayagamoorthy, 2008). Conversely, if a swarm is following a known target (Menda et al, 2010;Cummings et al, 2012) or defending a point from possibly hostile attackers (Cummings and Mitchell, 2008), the operator must act as a 'leader' and ensure the swarm is correctly following the target or defending against actual threats, requiring a more active operator (rather than a supervisor). Generally, data was gathered via self-assessment on cognitive workload, errors or time to task completion in a mission, or using fNIR to examine exactly which areas of the operators' brains were stimulated by given tasks.…”

“…These swarms used current technology and therefore had very rudimentary 'intelligence' and decision-making. Additionally, Menda et al (2010) found that cognitive workload increased rapidly when operators had to navigate UAVs using first person perspective, which reduced the number of UAVs that a single operator could simultaneously control successfully. This indicates that altering the viewpoint to third person and providing more general situational context can make operators more effective.…”

“…To assess cognitive workload, the researchers most often used self-assessments, such as the NASA TLX, or interviews with the participants. Menda et al (2010) and Hing and Oh (2010) used Functional Near-Infrared Spectroscopy (fNIR), in addition to surveys, to directly observe which regions of the brain were activated during specific swarm direction tasks.…”

“…This allows operators to use whichever command method is more intuitive for them and also provides redundancy in the event of a loss of one capability, such as a loud battlefield masking the sound of voice commands. Even if multimodal control is not possible, adding multiple types of feedback was also found to reduce cognitive workload (Menda et al, 2010), improve operator SA, and reduce navigation errors (Hing and Oh, 2010). By adding a "chase camera" that provides a "view of being towed behind the aircraft" (Hing andOh, 2010, p. 2510), operators were better able to spatially localize their UAVs using the wider field of view than those using the onboard cameras currently used by most UAVs (Menda et al, 2010).…”

“…Even if multimodal control is not possible, adding multiple types of feedback was also found to reduce cognitive workload (Menda et al, 2010), improve operator SA, and reduce navigation errors (Hing and Oh, 2010). By adding a "chase camera" that provides a "view of being towed behind the aircraft" (Hing andOh, 2010, p. 2510), operators were better able to spatially localize their UAVs using the wider field of view than those using the onboard cameras currently used by most UAVs (Menda et al, 2010). Adding motion simulation also improves operator navigation and lowers cognitive workload, as measured with fNIR (Hing and Oh, 2010).…”