Humans interacting with multi-robot systems: a natural affect-based approach

Abstract

This paper proposes a novel human-multi-robot-system interaction approach that enjoys two main features: natural interaction and affect-based adaptation of robots behavior. Specifically, the proposed system enables interaction by means of a wrist-worn device, such as a commercial smartwatch, which allows to track user's movements and heart activity. Thus, on the one side, the proposed system allows the user to intuitively drive the robots by establishing a natural mapping between wrist movements and robots velocity. On the other side, the system estimates user's mental fatigue during interaction by means of the analysis of heart rate variability. The proposed interaction system adapts then the behavior of the multi-robot system when the interacting user gets overwhelmed with the interaction and control task, which is then simplified. Experimental validation is provided, to show the effectiveness of the proposed system. First, the natural and affect-based interaction are considered separately. Then, the approach is tested considering a complex realistic scenario, which is simulated in virtual reality in order to get an immersive and realistic interaction experience. The results of the experimental validation clearly show that the proposed affect-based adaptive system leads to relieving the user's fatigue and mental workload. Keywords Human-multi-robot-system interaction • Natural ineraction • Affective robotics • Human-centred robotics This is one of the several papers published in Autonomous Robots comprising the Special Issue on Multi-Robot and Multi-Agent Systems.

[Retracted] Construction of a Multimodal Neuroimaging Data Fusion System and Evaluation of Mental Fatigue Using Nonlinear Analysis

“…e problem of mental fatigue becomes increasingly serious as market competition intensifies, and the pace of production and life accelerates. e aggravation of mental fatigue causes a series of problems, such as reduced work efficiency and traffic accidents [11]. erefore, the analysis of mental fatigue is necessary.…”

Towards evaluating the impact of swarm robotic control strategy on operators’ cognitive load

“…A first step of outlier removal, ectopic beats removal, and linear interpolation was conducted to prepare the remaining datasets. We used the SDNN measure (i.e., the standard deviation of the interval between normal heartbeats) to investigate cognitive load, as research has shown that SDNN decreases with increased cognitive load [33], [34] and that there is a positive correlation between SDNN and cognitive performance [35].…”

The design of self-organizing human–swarm intelligence

“…When walking down the street, you rarely think about walking. Therefore, the (cognitive) state of humans should be translated by the interface into commands (Alboul et al, 2008; Bales & Kong, 2017; Karavas, Larsson, & Artemiadis, 2017; L Mondada, Karim, & F Mondada, 2016; Podevijn et al, 2016; Pourmehr, Monajjemi, Vaughan, & Mori, 2013; Schwarz & Fuchs, 2017; Villani et al, 2020) for the swarm clusters. This is referred to as passive (Kolling et al, 2015) or implicit (Gildert, Millard, Pomfret, & Timmis, 2018) interaction, that is, unconscious control, which can be contrasted to active/explicit control (e.g.…”

“…His heartbeat and respiration accelerate-even the advanced homeostatic temperature control of his body armour cannot make Tom's physical and psychological stress go all away as he enters increasingly hazardous zones. With increasing stress (Schwarz & Fuchs, 2017), some robots that have been exploring the warehouse allocate to Tom (Villani, Capelli, Secchi, Fantuzzi, & Sabattini, 2020). It appears as if the robots sensed that Tom requires increased protection, as if they are part of Tom.…”