Visual-Inertial Navigation Algorithm Development Using Photorealistic Camera Simulation in the Loop

Sayre-McCord, Thomas; Guerra, Winter; Antonini, Amado; Arneberg, Jasper; Brown, A P Ian; Cavalheiro, Guilherme Venturelli; Fang, Yajun; Gorodetsky, Alex; Mccoy, D. S.; Quilter, Sebastian; Riether, Fabian; Tal, Ezra; Terzioğlu, Yunus; Carlone, Luca; Karaman, Sertaç

doi:10.1109/icra.2018.8460692

Cited by 36 publications

(33 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A recent work FlightGoggles [57], creates virtual reality environments for drones using the images streamed from the Unity3D game engine. However, for maximum realism, FlightGoggles requires a fully functioning drone that must fly during tests, with its sensory data being streamed in from the game engine.…”

Section: Related Workmentioning

confidence: 99%

MAVBench: Micro Aerial Vehicle Benchmarking

Boroujerdian

Genç

Krishnan

et al. 2018

2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs) are getting closer to becoming ubiquitous in everyday life. Among them, Micro Aerial Vehicles (MAVs) have seen an outburst of attention recently, specifically in the area with a demand for autonomy. A key challenge standing in the way of making MAVs autonomous is that researchers lack the comprehensive understanding of how performance, power, and computational bottlenecks affect MAV applications. MAVs must operate under a stringent power budget, which severely limits their flight endurance time. As such, there is a need for new tools, benchmarks, and methodologies to foster the systematic development of autonomous MAVs. In this paper, we introduce the "MAVBench" framework which consists of a closed-loop simulator and an end-to-end application benchmark suite. A closed-loop simulation platform is needed to probe and understand the intra-system (application data flow) and inter-system (system and environment) interactions in MAV applications to pinpoint bottlenecks and identify opportunities for hardware and software co-design and optimization. In addition to the simulator, MAVBench provides a benchmark suite, the first of its kind, consisting of a variety of MAV applications designed to enable computer architects to perform characterization and develop future aerial computing systems. Using our open source, end-to-end experimental platform, we uncover a hidden, and thus far unexpected compute to total system energy relationship in MAVs. Furthermore, we explore the role of compute by presenting three case studies targeting performance, energy and reliability. These studies confirm that an efficient system design can improve MAV's battery consumption by up to 1.8X.

show abstract

Section: Related Workmentioning

confidence: 99%

MAVBench: Micro Aerial Vehicle Benchmarking

Boroujerdian

Genç

Krishnan

et al. 2018

2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)

View full text Add to dashboard Cite

show abstract

“…Particularly in GPS-denied environments, cameras may be essential for effective state estimation. Visual inertial odometry (VIO) algorithms combine camera images with preintegrated IMU measurements to estimate the vehicle state [21], [24]. While these algorithms are often critical for safe navigation, it is challenging to verify their performance in varying conditions.…”

Section: A Aircraft-in-the-loop High-speed Flight Using Visual Inertmentioning

confidence: 99%

“…To successfully complete the entire track, the quadcopter had to pass all gates in order. Points were deducted for missed gates, leading to the following performance metric score = 10 · gates − time (21) where gates is the number of gates passed in order and time is the time taken in seconds to reach the final gate. If the final gate was not reached within the race time limit or the quadcopter collided with an environment object, a score of zero was recorded.…”

Section: Alphapilot Challengementioning

confidence: 99%

FlightGoggles: Photorealistic Sensor Simulation for Perception-driven Robotics using Photogrammetry and Virtual Reality

Guerra

Tal

Murali

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

107

View full text Add to dashboard Cite

FlightGoggles is a photorealistic sensor simulator for perception-driven robotic vehicles. The key contributions of FlightGoggles are twofold. First, FlightGoggles provides photorealistic exteroceptive sensor simulation using graphics assets generated with photogrammetry. Second, it also provides the ability to combine (i) synthetic exteroceptive measurements generated in silico in real time and (ii) vehicle dynamics and proprioceptive measurements generated in motio by vehicle(s) in flight in a motion-capture facility. FlightGoggles is capable of simulating a virtual-reality environment around autonomous vehicle(s) in flight. While a vehicle is in flight in the Flight-Goggles virtual reality environment, exteroceptive sensors are rendered synthetically in real time while all complex extrinsic dynamics are generated organically through the natural interactions of the vehicle. The FlightGoggles framework allows for researchers to accelerate development by circumventing the need to estimate complex and hard-to-model interactions such as aerodynamics, motor mechanics, battery electrochemistry, and behavior of other agents. The ability to perform vehiclein-the-loop experiments with photorealistic exteroceptive sensor simulation facilitates novel research directions involving, e.g., fast and agile autonomous flight in obstacle-rich environments, safe human interaction, and flexible sensor selection. Flight-Goggles has been utilized as the main test for selecting nine teams that will advance in the AlphaPilot autonomous drone racing challenge. We survey approaches and results from the top AlphaPilot teams, which may be of independent interest.

show abstract

“…Este teste consiste em fazer um drone passar por uma sequência de janelas que define o trajeto da corrida. O simulador utilizado foi o FlightGoggles, apresentado em Sayre-McCord et al (2018), desenvolvido por pesquisadores do MIT (Massachusetts Institute of Technology). O presente trabalho consiste na união de diversas técnicas, algumas já bem conhecidas naárea de robótica móvel, para a obtenção de uma solução para o desafio.…”

Section: Introductionunclassified

Planejamento de trajetória, localização e controle aplicados a um quadrirrotor autônomo de corrida

Rezende

Machado

Miranda

et al. 2019

Anais Do 14º Simpósio Brasileiro De Automação Inteligente

View full text Add to dashboard Cite

In this paper, we present a methodology to make an autonomous quadrotor fly through a sequence of gates only with on-board sensors. Our work is a solution to the AlphaPilot Challenge, proposed by the Lookheed Martin Company. In the challenge, the quadriprotor must be able to compete with human-piloted drones in a race. First, we propose a strategy to generate a smooth trajectory that passes through the gates. Then we developed a localization system, which merges image data from an on-board camera with IMU data. Finally, we present an artificial vector field based strategy used to control the quadrotor. Our results are validated with simulations in the official simulator of the competition. Resumo: Este artigo apresenta uma metodologia para guiar um quadrirrotor autônomo através de uma sequência de portões utilizando apenas sensores onboard. O trabalhoé uma solução para o desafio AlphaPilot, proposto pela companhia Lookheed Martin. No desafio, o quadrirrotor deve ser capaz de competir com drones pilotados por humanos em uma corrida. Primeiramente, uma estratégia para gerar uma trajetória suave que trafega através das janelasé proposta. Então, e desenvolvido um sistema de localização que une informações de imagens de uma câmera frontal com informações da IMU. Finalmente, uma estratégia baseada em campos vetoriais artificiais utilizada para controlar o quadrirrotoré apresentada. Os resultados são validados com simulações no simulador oficial da competição.

show abstract

Visual-Inertial Navigation Algorithm Development Using Photorealistic Camera Simulation in the Loop

Cited by 36 publications

References 16 publications

MAVBench: Micro Aerial Vehicle Benchmarking

MAVBench: Micro Aerial Vehicle Benchmarking

FlightGoggles: Photorealistic Sensor Simulation for Perception-driven Robotics using Photogrammetry and Virtual Reality

Planejamento de trajetória, localização e controle aplicados a um quadrirrotor autônomo de corrida

Contact Info

Product

Resources

About