Data-driven body–machine interface for the accurate control of drones

Abstract

SignificanceThe teleoperation of nonhumanoid robots is often a demanding task, as most current control interfaces rely on mappings between the operator’s and the robot’s actions, which are determined by the design and characteristics of the interface, and may therefore be challenging to master. Here, we describe a structured methodology to identify common patterns in spontaneous interaction behaviors, to implement embodied user interfaces, and to select the appropriate sensor type and positioning. Using this method, we developed an intuitive, gesture-based control interface for real and simulated drones, which outperformed a standard joystick in terms of learning time and steering abilities. Implementing this procedure to identify body-machine patterns for specific applications could support the development of more intuitive and effective interfaces.

The dynamics of motor learning through the formation of internal models

“…We investigated how users of a body machine interface learn to reorganize or “remap” their body motions as they practice controlling an external object through the BoMI. The controlled object could be a wheelchair, a robotic assistant, or a drone [15-17]. Here we focus on the control of a computer cursor whose two-dimensional coordinates determine its location on a computer screen.…”

“…A broad spectrum of sensors, such inertial measurement units (IMUs) placed on the head [37] or electroencephalography (EEG) systems [38], are available for detecting and decoding movement intentions. A body machine interface (BoMI) captures residual body motions by optical [17, 39, 40], accelerometric [15, 41], or electromyographic sensors [42], and maps the sensor signals onto commands for external devices such as powered wheelchairs [15] or drones [17], or onto computer inputs. At the other end of the spectrum, brain-machine interfaces decode motor intention from neural activity recorded in motor or premotor cortical areas [19, 20, 43].…”

“…In a first map, the subjects control the motions of their bodies by acting on a multitude of muscles and joints. In a second map, the signals triggered by these body motions determine the lower dimensional state of an external object such as a wheelchair [15], a cursor on a computer screen [16], or a drone [17]. Here we make the critical but reasonable assumption that the subjects have already acquired in a stable form the expertise needed to control the motion of their body, or at least portions of it that were unaffected by injury or disease.…”

Haptic Guidance with a Soft Exoskeleton Reduces Error in Drone Teleoperation

“…This wearable suit tracks the torso orientation, and converts it into drone commands. The design of the exosuit and its ergonomics are suited for this flight style that has been identified has a natural and intuitive approach that naïve users adopt to fly fixed wing drones [5]. The user sits on a backless stool and bends his torso forward and backward in the sagittal plane, to control the pitch up and down maneuvers respectively.…”

“…In order to make drones more accessible to non-expert users and facilitate their direct control in demanding tasks such as inspection or rescue missions, several studies have investigated the use of gestures [3], [4]. In a previous study, the authors have identified an intuitive upper body movement pattern that naïve users exploited to fly a fixed wing drone [5]. This embodied flight style, which allows the user to directly control the pitch and roll of a drone using torso movements, reduces learning time and increases performance when compared to the use of a traditional remote controllers.…”