A control architecture for multiple drones operated via multimodal interaction in search & rescue mission

Abstract

An architecture suitable for the control of multiple unmanned aerial vehicles deployed in Search & Rescue missions is presented in this paper. In the proposed system, a single colocated human operator is able to coordinate the actions of a set of robots in order to retrieve relevant information of the environment. This work is framed in the context of the SHERPA project whose goal is to develop a mixed ground and aerial robotic platform to support search and rescue activities in alpine scenario. Differently from typical human-drone interaction settings, here the operator is not fully dedicated to the drones, but involved in search and rescue tasks, hence only able to provide sparse and incomplete instructions to the robots. In this work, the domain, the interaction framework and the executive system for the autonomous action execution are discussed. The overall system has been tested in a real world mission with two drones equipped with on-board cameras

The current state and future outlook of rescue robotics

“…The SHERPA project approached this problem by introducing the “busy genius”—a rescuer colocated with robots and equipped with a set of wearable devices for multimodal interaction (Marconi et al, ). Since the rescuer is also busy with other activities the interaction happens sporadically and relies on a mixed‐initiative system (Cacace et al, ), where the the mission planner utilizes delegation (Doherty, Heintz, & Kvarnström, ) to distribute tasks to a potentially heterogeneous team of agents. Further extending the concept of wearable interfaces, Wang et al () developed an exoskeleton for the whole‐body human‐in‐the‐loop teleoperation of a humanoid robot for SAR.…”

“…When the operator is deployed alongside the robot and shares its environment, one may use instead proximity interaction modalities, that assume that a direct line‐of‐sight to the robot is available; then different interfaces can be used, ranging from standard joysticks (e.g., for low‐level control of UAVs) to hands‐free gesture‐based interfaces based on sensorized armbands (Wolf, Assad, Vernacchia, Fromm, & Jethani, ), armbands (Cacace et al, ; Gromov, Gambardella, & Giusti, ), smart watches (Villani et al, ) or voice commands (Gromov, Gambardella, & Di Caro, ).…”

A Semi-Physical Platform for Guidance and Formations of Fixed-Wing Unmanned Aerial Vehicles

“…The availability of low-cost sensors, electronics, and air-frames has promoted a significant interest in Unmanned Aerial Vehicles (UAVs) among aircraft hobbyists, academic researchers, and industries [1,2]. Among the different families of UAVs, the fixed-wing type is generally deployed when extensive areas are to be covered in a short time.…”

“…Parametric uncertainties and unmodelled dynamics will certainly appear in the UAV structure and cannot be captured by point mass kinematic models [27]. 2 The actual path-following performance does not depends only on the commanded course angle. The autopilot in charge of regulating roll, pitch and altitude (rudder/wing/aileron actuators) crucially contributes to the final performance [28].…”

SwarmTouch: Guiding a Swarm of Micro-Quadrotors With Impedance Control Using a Wearable Tactile Interface

“…Human-swarm interaction (HSI) combines many research topics, which are well described in [4], and could vary from communication channels to a level of swarm autonomy. The authors in [5] presented a multimodal interaction strategy between a human and a formation of drones for search and rescue operations. Gestures and speech recognition along with a tablet allowed the user to control the fleet of quadrotors.…”