Reinforcement learning (RL) is an attractive way to implement high-level decision-making policies for autonomous driving, but learning directly from a real vehicle or a high-fidelity simulator is variously infeasible. We therefore consider the problem of transfer reinforcement learning and study how a policy learned in a simple environment using
WiseMove
can be transferred to our high-fidelity simulator, W
ise
M
ove
.
WiseMove
is a framework to study safety and other aspects of RL for autonomous driving. W
ise
M
ove
accurately reproduces the dynamics and software stack of our real vehicle.
We find that the accurately modelled perception errors in W
ise
M
ove
contribute the most to the transfer problem. These errors, when even naively modelled in
WiseMove
, provide an RL policy that performs better in W
ise
M
ove
than a hand-crafted rule-based policy. Applying domain randomization to the environment in
WiseMove
yields an even better policy. The final RL policy reduces the failures due to perception errors from 10% to 2.75%. We also observe that the RL policy has significantly less reliance on velocity compared to the rule-based policy, having learned that its measurement is unreliable.
Machine learning can provide efficient solutions to the complex problems encountered in autonomous driving, but ensuring their safety remains a challenge. A number of authors have attempted to address this issue, but there are few publicly-available tools to adequately explore the trade-offs between functionality, scalability, and safety. We thus present WiseMove, a software framework to investigate safe deep reinforcement learning in the context of motion planning for autonomous driving. WiseMove adopts a modular learning architecture that suits our current research questions and can be adapted to new technologies and new questions. We present the details of WiseMove, demonstrate its use on a common traffic scenario, and describe how we use it in our ongoing safe learning research. * contributed equally 1 uwaterloo.ca/waterloo-intelligent-systems-engineering-lab/ arXiv:1902.04118v1 [cs.LG]
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.