Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning

Chalaki, Behdad; Beaver, Logan E.; Remer, Ben; Jang, Kathy; Vinitsky, Eugene; Bayen, Alexandre M.; Malikopoulos, Andreas A.

doi:10.1109/icca51439.2020.9264552

Cited by 33 publications

(22 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Adversarial attacks can be performed manually [12] [20] by using hardcoded out-of-distribution images added to autonomous driving environments. Attacks can also be added intelligently using adversarial machine learning [19], where noise perturbations are mixed within the input images of the driving models in order to predict and analyze steering angles. Among the latter category, there are several types of approaches proposed.…”

Section: Related Workmentioning

confidence: 99%

“…RL Approaches. Recent work [19] proposes to use RL based driving agents to test connected cars by perturbing both the inputs and outputs of an AC during training. However, this approach targets a mixed-traffic driving with a single AC and multiple human-driven cars, thus it does not consider complex scenario having more than one non-communicating AC agents.…”

Section: Related Workmentioning

confidence: 99%

“…However, while RL has shown great results as an adversarial agent, RL is still mainly used for testing ACs in a single-agent environment. Recent work [19] made progress towards testing connected and automated vehicles by injecting adversarial noise in a mixed-traffic-based network environment. However, this work is of limited practical use as it does not consider multiple non-communicating AC agents.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies

Sharif¹,

Marijan²

2021

Preprint

View full text Add to dashboard Cite

Deep reinforcement learning is widely used to train autonomous cars in a simulated environment. Still, autonomous cars are well known for being vulnerable when exposed to adversarial attacks. This raises the question of whether we can train the adversary as a driving agent for finding failure scenarios in autonomous cars, and then retrain autonomous cars with new adversarial inputs to improve their robustness. In this work, we first train and compare adversarial car policy on two custom reward functions to test the driving control decision of autonomous cars in a multi-agent setting. Second, we verify that adversarial examples can be used not only for finding unwanted autonomous driving behavior, but also for helping autonomous driving cars in improving their deep reinforcement learning policies. By using a high fidelity urban driving simulation environment and vision-based driving agents, we demonstrate that the autonomous cars retrained using the adversary player noticeably increase the performance of their driving policies in terms of reducing collision and offroad steering errors.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies

Sharif¹,

Marijan²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This cutting-edge technology is becoming a popular tool to curb the emission across the world. Many educational institutions, such as the University of Delaware, USA, have autonomous vehicle test-bed facilities to experiment with this technology [22]. In addition to testing the above scenarios separately, this study also provides insights into different combinations of these scenarios.…”

Section: Introductionmentioning

confidence: 99%

Simulating the Impacts of Hybrid Campus and Autonomous Electric Vehicles as GHG Mitigation Strategies: A Case Study for a Mid-Size Canadian Post-Secondary School

Saha

Fatmi

2021

Sustainability

View full text Add to dashboard Cite

This paper presents how a post-secondary institution like University of British Columbia’s Okanagan (UBCO) campus can reduce its carbon footprint and be aligned with the government’s target through promoting virtual campus and autonomous electric vehicles (AEVs). Different virtual campus scenarios are developed: online classes only, working-from-home only, and a hybrid of both. In the case of AEVs, alternative penetration rates for levels 2 and 5 are considered. A total of 50 scenarios are tested using a sub-area transport simulation model for UBCO, which is extracted from the regional travel demand forecasting model. The results suggest that a 40% AEV penetration rate coupled with fully in-person classes reduces GHG by ~36% compared to the 2018-level, which will help UBCO to achieve their 2030 emission reduction target and be aligned with the provincial target. The 50% AEV and 10% hybrid virtual campus reduces emissions by ~48%, which is aligned with the 2040 provincial target. A fully virtual campus will help to reach the 2050 provincial target by reducing GHG by ~76%. The results further demonstrate that level 5 AEVs produce lesser emissions than level 2 at a lower AEV penetration rate for the fully in-person campus scenario. At higher penetration rates, level 5 performs better only if it is coupled with 10% of students, faculties and staffs attending virtual campus scenario.

show abstract

“…In the area of traffic engineering, microscopic simulators have embedded with various traffic models and capture the dynamics of drivers as well as their interaction with infrastructure (Osorio, 2010). A simulation‐based method has been successfully applied for autonomous vehicle and traffic control problems (Chalaki et al., 2019; Jang et al., 2019). For road networks, a simulator could help generate large datasets and thus provides a great opportunity for solving the scarcity problem of OD flow sample.…”

Section: Introduction and Reviewmentioning

confidence: 99%

Dynamic origin‐destination flow estimation using automatic vehicle identification data: A 3D convolutional neural network approach

Tang

Cao

Chen

et al. 2020

Computer aided Civil Eng

View full text Add to dashboard Cite

Dynamic origin-destination (OD) flow estimation is one of the most fundamental problems in traffic engineering. Despite numerous existing studies, the OD flow estimation problem remains challenging, as there is large dimensional difference between the unknown values to be estimated and the known traffic observations. To meet the needs of active traffic management and control, accurate time-dependent OD flows are required to understand time-of-day traffic flow patterns. In this work, we propose a three-dimensional (3D) convolution-based deep neural network, "Res3D," to learn the high-dimensional correlations between local traffic patterns presented by automatic vehicle identification observations and OD flows. In this paper, a practical framework combining simulation-based model training and few-shot transfer learning is introduced to enhance the applicability of the proposed model, as continuously observing OD flows could be expensive. The proposed model is extensively tested based on a realistic road network, and the results show that for significant OD flows, the relative errors are stable around 5%, outperforming several other models, including prevalent neural networks as well as existing estimation models. Meanwhile, corrupted and out-of-distribution samples are generated as real-world samples to validate Res3D's transferability, and the results indicated a 60% improvement with few-shot transfer learning. Therefore, this proposed framework could help to bridge the gaps between traffic simulations and empirical cases.

show abstract

Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning

Cited by 33 publications

References 20 publications

Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies

Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies

Simulating the Impacts of Hybrid Campus and Autonomous Electric Vehicles as GHG Mitigation Strategies: A Case Study for a Mid-Size Canadian Post-Secondary School

Dynamic origin‐destination flow estimation using automatic vehicle identification data: A 3D convolutional neural network approach

Contact Info

Product

Resources

About