AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles

Shah, Shital; Dey, Debadeepta; Lovett, Chris; Kapoor, Ashish

doi:10.1007/978-3-319-67361-5_40

Cited by 1,407 publications

(868 citation statements)

References 14 publications

Supporting

Mentioning

755

Contrasting

Unclassified

Order By: Relevance

“…A change of the camera position in the vehicle could lead to the significant systematic errors because certain combinations of rotation and translation are not presented in the training set. One way to overcome these issues is the use of synthetic data collected from autonomous driving simulators like [7,28,24]. A simulator allows to attach one or multiple cameras at specific locations on the vehicle, vary weather, time and lighting con- ditions, and define custom camera intrinsic parameters.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…We chose the CARLA simulator [7] to generate synthetic training data out of convenience as it facilitates an autopilot mode. Other simulators ( [28,24] might be applicable instead or in addition to achieve an ever larger variability of the data. The CARLA simulator allows the time of day and weather conditions to be changed-in particular, we can add puddles and vary rain intensity-and provides six different maps.…”

Section: Training Detailsmentioning

confidence: 99%

See 1 more Smart Citation

Estimation of Absolute Scale in Monocular SLAM Using Synthetic Data

Rukhovich¹

2020

PRIN

View full text Add to dashboard Cite

This paper addresses the problem of scale estimation in monocular SLAM by estimating absolute distances between camera centers of consecutive image frames. These estimates would improve the overall performance of classical (not deep) SLAM systems and allow metric feature locations to be recovered from a single monocular camera. We propose several network architectures that lead to an improvement of scale estimation accuracy over the state of the art. In addition, we exploit a possibility to train the neural network only with synthetic data derived from a computer graphics simulator. Our key insight is that, using only synthetic training inputs, we can achieve similar scale estimation accuracy as that obtained from real data. This fact indicates that fully annotated simulated data is a viable alternative to existing deep-learning-based SLAM systems trained on real (unlabeled) data. Our experiments with unsupervised domain adaptation also show that the difference in visual appearance between simulated and real data does not affect scale estimation results. Our method operates with low-resolution images (0.03 MP), which makes it practical for real-time SLAM applications with a monocular camera.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Training Detailsmentioning

confidence: 99%

Estimation of Absolute Scale in Monocular SLAM Using Synthetic Data

Rukhovich¹

2020

PRIN

View full text Add to dashboard Cite

show abstract

“…Autonomous driving simulators are often used to test autonomous vehicles for the sake of efficiency and safety [30]- [33]. After testing popular autonomous simulators [34]- [37], we chose to run our experiments on the CARLA [8] (CAR Learning to Act) autonomous vehicle simulator, due to its feature-set and ease of source code modification .…”

Section: A Autonomous Vehicle Simulatormentioning

confidence: 99%

Attacking vision-based perception in end-to-end autonomous driving models

Boloor

Garimella

et al. 2020

Journal of Systems Architecture

View full text Add to dashboard Cite

Recent advances in machine learning, especially techniques such as deep neural networks, are enabling a range of emerging applications. One such example is autonomous driving, which often relies on deep learning for perception. However, deep learning-based perception has been shown to be vulnerable to a host of subtle adversarial manipulations of images. Nevertheless, the vast majority of such demonstrations focus on perception that is disembodied from end-to-end control. We present novel endto-end attacks on autonomous driving in simulation, using simple physically realizable attacks: the painting of black lines on the road. These attacks target deep neural network models for endto-end autonomous driving control. A systematic investigation shows that such attacks are easy to engineer, and we describe scenarios (e.g., right turns) in which they are highly effective. We define several objective functions that quantify the success of an attack and develop techniques based on Bayesian Optimization to efficiently traverse the search space of higher dimensional attacks. Additionally, we define a novel class of hijacking attacks, where painted lines on the road cause the driverless car to follow a target path. Through the use of network deconvolution, we provide insights into the successful attacks, which appear to work by mimicking activations of entirely different scenarios. Our code is available on https://github.com/xz-group/AdverseDrive Index Terms-machine learning, adversarial examples, autonomous driving, end-to-end learning, bayesian optimization

show abstract

“…We created the avatar within the AirSim [25] environment using Unreal Engine 4 (UE4) [8]. The avatar, a head, and torso is placed in a room with multiple light sources (windows, overhead lights, and spotlights).…”

Section: The Avatarmentioning

confidence: 99%

A High-Fidelity Open Embodied Avatar with Lip Syncing and Expression Capabilities

Aneja

McDuff

Shah

2019

2019 International Conference on Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

Embodied avatars as virtual agents have many applications and provide benefits over disembodied agents, allowing nonverbal social and interactional cues to be leveraged, in a similar manner to how humans interact with each other. We present an open embodied avatar built upon the Unreal Engine that can be controlled via a simple python programming interface. The avatar has lip syncing (phoneme control), head gesture and facial expression (using either facial action units or cardinal emotion categories) capabilities. We release code and models to illustrate how the avatar can be controlled like a puppet or used to create a simple conversational agent using public application programming interfaces (APIs). GITHUB link: https://github.com/danmcduff/AvatarSim Figure 1: We present an open embodied avatar with lip syncing and expression capabilities that can be controlled via simple python interface. We provide examples of how to combine this with publicly available speech and dialogue APIs to construct a conversational embodied agent.

show abstract

AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles

Cited by 1,407 publications

References 14 publications

Estimation of Absolute Scale in Monocular SLAM Using Synthetic Data

Estimation of Absolute Scale in Monocular SLAM Using Synthetic Data

Attacking vision-based perception in end-to-end autonomous driving models

A High-Fidelity Open Embodied Avatar with Lip Syncing and Expression Capabilities

Contact Info

Product

Resources

About