Multi-Robot Interfaces and Operator Situational Awareness: Study of the Impact of Immersion and Prediction

Abstract

Multi-robot missions are a challenge for operators in terms of workload and situational awareness. These operators have to receive data from the robots, extract information, understand the situation properly, make decisions, generate the adequate commands, and send them to the robots. The consequences of excessive workload and lack of awareness can vary from inefficiencies to accidents. This work focuses on the study of future operator interfaces of multi-robot systems, taking into account relevant issues such as multimodal interactions, immersive devices, predictive capabilities and adaptive displays. Specifically, four interfaces have been designed and developed: a conventional, a predictive conventional, a virtual reality and a predictive virtual reality interface. The four interfaces have been validated by the performance of twenty-four operators that supervised eight multi-robot missions of fire surveillance and extinguishing. The results of the workload and situational awareness tests show that virtual reality improves the situational awareness without increasing the workload of operators, whereas the effects of predictive components are not significant and depend on their implementation.

Comprehensive architecture for intelligent adaptive interface in the field of single-human multiple-robot interaction

“…Our results Results of [24] Results of [25] Results of [26] Results of [27] Results of [28] F I G U R E 1 6 Comparison between NASA task load index (NASA-TLX) percentage decrease in our results and reported results of somewhat similar experiments TLX. Furthermore, in [24], it has been reported that the perceived SA for controlling five UAVs is equal to 5.3.…”

“…As depicted in Figure 17, our result has a good balance in terms of increasing SAGAT and decreasing NASA- Our results Results of [24] Results of [25] Results of [26] Results of [27] Results of [28] 19.57%…”

Analyzing and improving multi-robot missions by using process mining

“…Some important issues discovered in the multirobot missions were the causes of accident, the inefficiencies in the management of resources and the limitations of the commanding interface. In fact, the analysis performed in this work was the starting point for the design of an adaptive and immersive interface in a later one (Roldán et al 2017b).…”

Press Start to Play: Classifying Multi-Robot Operators and Predicting Their Strategies through a Videogame

“…The challenges faced by multi-robot operators can be addressed by developing immersive and adaptive interfaces. The former apply virtual or augmented reality to introduce operators into the environment, improving their perception of the agents and their mission awareness [6]. The latter take on some of the operator's tasks and provide the worker with the most relevant information and the most suitable decisions available, thus reducing the operators' workload and improving their performance [7].…”

“…Some of the most promising strategies to address these issues involve the development of adaptive interfaces: selection of relevant information, adaptation to operator preferences, and guidance of operator attention [11]. In previous works, the positive impact of immersion and prediction on operator workload, situational awareness, and performance was demonstrated in a mission with multiple drones [6] and another mission with ground, aerial, and manipulator robots [12].…”