Autonomous, Vision‐based Flight and Live Dense 3D Mapping with a Quadrotor Micro Aerial Vehicle

Abstract

The use of mobile robots in search-and-rescue and disaster-response missions has increased significantly in recent years. However, they are still remotely controlled by expert professionals on an actuator set-point level, and they would benefit, therefore, from any bit of autonomy added. This would allow them to execute highlevel commands, such as "execute this trajectory" or "map this area." In this paper, we describe a vision-based quadrotor micro aerial vehicle that can autonomously execute a given trajectory and provide a live, dense three-dimensional (3D) map of an area. This map is presented to the operator while the quadrotor is mapping, so that there are no unnecessary delays in the mission. Our system does not rely on any external positioning system (e.g., GPS or motion capture systems) as sensing, computation, and control are performed fully onboard a smartphone processor. Since we use standard, off-the-shelf components from the hobbyist and smartphone markets, the total cost of our system is very low. Due to its low weight (below 450 g), it is also passively safe and can be deployed close to humans. We describe both the hardware and the software architecture of our system. We detail our visual odometry pipeline, the state estimation and control, and our live dense 3D mapping, with an overview of how all the modules work and how they have been integrated into the final system. We report the results of our experiments both indoors and outdoors. Our quadrotor was demonstrated over 100 times at multiple trade fairs, at public events, and to rescue professionals. We discuss the practical challenges and lessons learned. Code, datasets, and videos are publicly available to the robotics community. C 2015 Wiley Periodicals, Inc. SUPPLEMENTARY MATERIALThis paper is accompanied by videos demonstrating the capabilities of our platform in outdoor and indoor scenarios: r Indoor evaluation (disturbance and autonomous, visionbased live 3D mapping): http://youtu.be/sdu4w8r_fWc r Outdoor autonomous, vision-based flight over disaster zone: http://youtu.be/3mNY9-DSUDk r Outdoor autonomous, vision-based flight with live 3D mapping: http://youtu.be/JbACxNfBI30More videos can be found on our Youtube channel: https://www.youtube.com/user/ailabRPG/videos Our visual odometry code (called SVO) for visionbased navigation has been released open source and is freely available on the authors' homepage. * The authors are with the Robotics and Perception Group, University of Zurich, Switzerland-http://rpg.ifi.uzh.ch.

Dynamic collaboration without communication: Vision-based cable-suspended load transport with two quadrotors

“…We send this command to the quadrotors onboard controller [18], which takes care of controlling the attitude and thrust to the desired values.…”

“…Visual-inertial state estimation is achieved using a pipeline developed in-house [18]. The pipeline uses ROS as middleware, SVO [22] for VO and MSF [23] for fusion of VO and IMU into a visual-inertial odometry (VIO).…”

Simultaneous State Initialization and Gyroscope Bias Calibration in Visual Inertial Aided Navigation

“…• The loosely-coupled visual-inertial algorithm (MSF) [23] using pose estimates from the Semi-direct Visual Odometry (SVO) package [6] (how to combine MSF with SVO can be found in [18]). The reason we included SVO+MSF in the validation is to have a reference state-of-the-art pose estimation method.…”

“…These techniques, known as Structure from Motion, can recover the relative rotation and translation up to an unknown scale factor between two camera poses [16]. Such methods are currently used in state-of-the-art visual navigation methods for MAVs to initialize maps [6,17,18]. However, the knowledge of the absolute scale, and, at least, of the absolute roll and pitch angles, is essential for many applications ranging from autonomous navigation in GPS-denied environments to 3D reconstruction and augmented reality.…”

“…We use the same MAV used in [18], Section 3.4. Specifically, our quadrotor relies on the frame of the Parrot AR.Drone 2.0 including their motors, motor controllers, gears, and propellers.…”