Image-based estimation, planning, and control for high-speed flying through multiple openings

Abstract

This article focuses on enabling an aerial robot to fly through multiple openings at high speed using image-based estimation, planning, and control. State-of-the-art approaches assume that the robot’s global translational variables (e.g., position and velocity) can either be measured directly with external localization sensors or estimated onboard. Unfortunately, estimating the translational variables may be impractical because modeling errors and sensor noise can lead to poor performance. Furthermore, monocular-camera-based pose estimation techniques typically require a model of the gap (window) in order to handle the unknown scale. Herein, a new scheme for image-based estimation, aggressive-maneuvering trajectory generation, and motion control is developed for multi-rotor aerial robots. The approach described does not rely on measurement of the translational variables and does not require the model of the gap or window. First, the robot dynamics are expressed in terms of the image features that are invariant to rotation (invariant features). This step decouples the robot’s attitude and keeps the invariant features in the flat output space of the differentially flat system. Second, an optimal trajectory is efficiently generated in real time to obtain the dynamically-feasible trajectory for the invariant features. Finally, a controller is designed to enable real-time, image-based tracking of the trajectory. The performance of the estimation, planning, and control scheme is validated in simulations and through 80 successful experimental trials. Results show the ability to successfully fly through two narrow openings, where the estimation and planning computation and motion control from one opening to the next are performed in real time on the robot.

Robots Under COVID-19 Pandemic: A Comprehensive Survey

“…The higher risk of malfunction blocks the wider application of drones during delivery operations, with their battery life shortened. Presently, drones are not able to fly in heavy rain or in extreme wind, have limited battery capacity, and significantly rely on satellite-based positioning systems for trajectory tracking [185] . Variability is one of the biggest challenges of cobots in workspaces.…”

Quadrotor going through a window and landing: An image-based visual servo control approach

“…There are several examples in the literature of recent work dedicated to the problem of flying autonomous vehicles in complex environment using vision systems. In Falanga et al (2017), Guo & Leang (2020), Loianno et al (2017), the authors specifically address the problem of going through a window using only a single camera and an IMU. However, estimation of vehicle's position and velocity is required in Falanga et al (2017), Loianno et al (2017).…”

“…Besides, the pose of the window is assumed to be known in Loianno et al (2017). Although the work in Guo & Leang (2020) directly uses image feature as position cue, estimates of the image depth are still required and the velocity vector is assumed to be known. In general, state estimation adds computational complexity, and the output is often sensitive to image noise and camera calibration errors.…”

Image-Based Visual Servoing of Rotorcrafts to Planar Visual Targets of Arbitrary Orientation

“…After the seminal work in [8], several improved DIBVS laws are proposed based on the spherical image moments, such as the works in [9]- [13]. However, the ill conditioning of the spherical moment features still has not been completely solved though some new image features based on spherical projection are proposed to alleviate the ill conditioning [6], [14].…”