Real-time dense surface reconstruction for aerial manipulation

Abstract

With robotic systems reaching considerable maturity in basic self-localization and environment mapping, new research avenues open up pushing for interaction of a robot with its surroundings for added autonomy. However, the transition from traditionally sparse feature-based maps to dense and accurate scene-estimation imperative for realistic manipulation is not straightforward. Moreover, achieving this level of scene perception in real-time from a computationally constrained and highly shaky and agile platform, such as a small an Unmanned Aerial Vehicle (UAV) is perhaps the most challenging scenario for perception for manipulation. Drawing inspiration from otherwise computationally constraining Computer Vision techniques, we present a system combining visual, inertial and depth information to achieve dense, local scene reconstruction of high precision in real-time. Our evaluation testbed is formed using ground-truth not only in the pose of the sensor-suite, but also the scene reconstruction using a highly accurate laser scanner, offering unprecedented comparisons of scene estimation to ground-truth using real sensor data. Given the lack of any real, ground-truth datasets for environment reconstruction, our V4RL Dense Surface Reconstruction dataset is publicly available 1 .

Keyframe-based Direct Thermal–Inertial Odometry

“…A forward Euler integration scheme is used to calculated the predicted IMU state. Given the re-projection and inertial measurement error equations, the cost function for the joint thermal-inertial problem can be written as: (8) where k represents the frame being processed in the sliding window K containing temporal frames and key-frames, L(k) represents the set of landmarks observable in frame k, e reproj represents the stacked vector of re-projection errors of every landmark l visible in frame k, W k,l reproj represents the co-variance matrix for landmark measurements in frame k, and W k imu represents the co-variance matrix for IMU measurements. This cost function is then minimized using the Google Ceres [26] optimization framework and produces an estimate of the robot pose.…”

Overview obstacle maps for obstacle‐aware navigation of autonomous drones

“…Using OKVIS (Leutenegger et al, , ) as VIO front‐end, the research work (Oleynikova, Burri, Lynen, & Siegwart, ) also proposes a method to localize a drone and a ground robot on the same map by means of a previously acquired reference map. Recently, a drone dense‐reconstruction (Karrer, Kamel, Siegwart, & Chli, ) data set has been released, which focuses on small working areas and producing precise 3D dense models for the purpose of object manipulation, in which ground‐truth position data acquired with a Leica Total Station are available.…”

DEPTH CAMERAS ON UAVs: A FIRST APPROACH

“…In the robotics community, while depth cameras are being used for indoor mapping applications frequently in the recent years (Henry et al, 2010;Endres et al, 2012;Kerl et al, 2013), there is also a variety of research projects testing SLAM techniques for UAV equipped with laser scanners (Zhang and Singh, 2014) or depth cameras (Bachrach et al, 2014;Loianno et al, 2015;Karrer et al, 2016;Huang et al, 2017). On the other hand, although depth cameras have been tested for their performance in cultural heritage applications (Wenzel et al, 2012;Cappelletto et al, 2016), up to our knowledge, there is no much published work on using depth cameras mounted on UAV for cultural heritage applications.…”

“…Such setups on UAVs are relatively novel and currently being tested for their performance (Karrer et al, 2016). As a case study for our experiments, a 16th century turbe (tomb) located in Trikala, central Greece was chosen (Figure 2b and 2c).…”