Extreme Scalability: Designing Interfaces and Algorithms for Soldier-Robotic Swarm Interaction

Abstract

Vehicle Technology Directorate, ARLApproved for public release; distribution unlimited. ii REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, ARL-TR-4800 SPONSOR/MONITOR'S ACRONYM(S) 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) SPONSOR/MONITOR'S REPORT NUMBER(S) DISTRIBUTION/AVAILABILITY STATEMENTApproved for public release; distribution unlimited. SUPPLEMENTARY NOTES ABSTRACTIn theory, autonomous robotic swarms can be used for critical Army tasks (i.e., accompanying convoys); however, the Soldier controlling the swarm must be able to monitor swarm status and correct actions, especially in disrupted or degraded conditions. For this two-year Director's Research Initiative (DRI), we designed metacognition algorithms and Soldier-swarm display concepts to allow Soldiers to efficiently interact with a robotic swarm participating in a representative convoy mission. We used a potential field approach for swarm control because it scales easily to large heterogeneous swarms and allows users to dynamically alter swarm behavior by adjusting field parameters. The Soldier-swarm interface displayed swarm and convoy geospatial position; swarm health and communication; and convoy status information, using visual, auditory, and tactile combinations. We measured swarm metacognition by determining the proportion of time the simulated swarm could maintain a specific orbital ring around the convoy over six terrains in 13-min scenarios. We tested interface effectiveness in a laboratory study using 16 male Marines (volunteers) with a mean age of 19 years. The metacognition results showed that the swarm could maintain the pre-defined dispersion more than 85% of the time in each terrain. Using multimodal displays, Soldier workload decreased and performance increased (i.e., response time reduced).

Temporal binding of multimodal controls for dynamic map displays

“…As with Haas et al [3], we found that temporal binding time for touch-first commands was significantly shorter than for speech-first commands, no matter what the level of workload. However, temporal binding time for speech-first commands was perceived to be two times greater than touch-first commands in the presence of the secondary task, while this difference was less in conditions with no workload.…”

“…Other researchers [2] found that multimodal controls for dynamic maps allows users to choose which input modality they want to use, in which order, when inputting information. Haas, Fields, Stachowiak, Hill and Pillalamarri [3] explored the use of multimodal touch and speech controls in a dynamic map display to allow U. S. Army Soldiers to maintain supervisory control of a simulated robotic swarm (a large number of small, relatively simple robots capable of autonomous travel and operation) to provide sentry protection to a simulated convoy of four vehicles. These researchers found that multimodal speech and touch control allowed efficient Soldier supervisory control of a dynamic map display showing swarm and convoy spatial information.…”

“…The research described in the previous paragraph [1][2][3] has a "human-centered" viewpoint, which explores the efficiency of human performance in a human-computer interface. However, few researchers have explored a system-centered viewpoint, which examines the effects of human output on enabling the computer system to be more adaptable to multimodal user command [4].…”

Human Interaction With Robot Swarms: A Survey

“…Besides displays, multimodal feedback to the operator has also been investigated [98]. Here, the authors used a potential field approach for controlling the swarm for a convoy protection scenario and designed an interface that provides feedback regarding the swarm speed, strength, capability, and dispersion.…”

Workload, awareness and automation in multiple-robot supervision

“…For example, a study on soldier-robotic swarm interactions suggested that use of multimodal (visual, auditory and tactile) display techniques influences the effectiveness and efficiency of supervisors. 33 The level of autonomy available in the robotic system can affect supervisor workload as well. 2,34 Automation is often employed to address the problem of high workload.…”