Collaborative 6DoF Relative Pose Estimation for Two UAVs with Overlapping Fields of View

Abstract

Driven by the promise of leveraging the benefits of collaborative robot operation, this paper presents an approach to estimate the relative transformation between two small Unmanned Aerial Vehicles (UAVs), each equipped with a single camera and an inertial sensor, comprising the first step of any meaningful collaboration. Formation flying and collaborative object manipulation are some of the few tasks that the proposed work has direct applications on, while forming a variablebaseline stereo rig using two UAVs carrying a monocular camera each promises unprecedented effectiveness in collaborative scene estimation.Assuming an overlap in the UAVs' fields of view, in the proposed framework, each UAV runs monocular-inertial odometry onboard, while an Extended Kalman Filter fuses the UAVs' estimates and common image measurements to estimate the metrically scaled relative transformation between them, in realtime. Decoupling the direction of the baseline between the cameras of the two UAVs from its magnitude, this work enables consistent and robust estimation of the uncertainty of the relative pose estimation. Our evaluation on both on simulated data and benchmarking datasets consisting of real aerial data, reveals the power of the proposed methodology in a variety of scenarios. Videohttps://youtu.be/Amkk8X826oI

Multiple UAV Systems: A Survey

“…For example, in [17], authors presented a collaborative localization framework for a team of micro aerial vehicles equipped with forward-facing monocular cameras, and tested it within Microsoft AirSim. Or [16] where authors tested collaborative relative pose estimation for two UAVs using both simulated and real publicly available datasets.…”

“…For example, in a paper [76], that discussed a surveillance system performing real-time video stitching from several sources, there could not be more than 12 UAVs due to some internal limitations of the system. And in another work [16], it was specifically described about having just two UAVs for a relative pose estimation with overlapping field of view.…”

“…Most of them do not report on performing a real-life experiment. Some authors, for example, work with prerecorded videos as in [16] that presented a work on relative pose estimation of a pair of UAVs that have overlapping fields of view, or, as in [17], working on monocular vision-based collaborative localization where their algorithm was tested offline on a pre-recorded data. In a paper [18] where the authors developed a framework for multiple UAV cooperative mapping and control with image based visual servoing, a map full of feature points was simulated.…”

Distributed Variable-Baseline Stereo SLAM from two UAVs

“…Other methods deal with creating a stereo baseline that is as large as possible by placing the two cameras at the tips of a fixedwing aircraft [7], [8], which still bears the limitation of the aircraft's dimensions. Attempting to push the robustness of stereo-based scale estimation further, the works in [1] and [10] investigate the estimation of the relative transformation between two independently moving UAVs equipped with monocular cameras exploiting of their (relative) motion and view overlap. However, as with [17], these methods solely rely on motion (i.e.…”

Flexible Trinocular: Non-rigid Multi-Camera-IMU Dense Reconstruction for UAV Navigation and Mapping

“…Achtelik et al [16] proposed an efficient EKF formulation that fuses relative IMU and vision measurements for stereo vision with two quadrocopters that have an overlapping field of view. Recently, this EKF formulation has been used in [1], [17], [18] for various applications. In [1], the initial concept of Flexible Stereo was introduced, further constraining the EKF formulation with a probabilistic wing model.…”