Visual-Inertial Telepresence for Aerial Manipulation

Abstract

This paper presents a novel telepresence system for enhancing aerial manipulation capabilities. It involves not only a haptic device, but also a virtual reality that provides a 3D visual feedback to a remotely-located teleoperator in real-time. We achieve this by utilizing onboard visual and inertial sensors, an object tracking algorithm and a pregenerated object database. As the virtual reality has to closely match the real remote scene, we propose an extension of a marker tracking algorithm with visual-inertial odometry. Both indoor and outdoor experiments show benefits of our proposed system in achieving advanced aerial manipulation tasks, namely grasping, placing, force exertion and peg-in-hole insertion.

Learning Multiplicative Interactions with Bayesian Neural Networks for Visual-Inertial Odometry

“…In this sense, purely a multi-modal learning approaches can also be useful for such advanced methods beyond end-to-end strategy. For future works, further evaluations on other public datasets (Sun et al, 2019;Caesar et al, 2019) and other applications of VIO (Lutz et al, 2020;Lee et al, 2020a) can be envisioned.…”