DeepRacer: Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning

Abstract

DeepRacer is a platform for end-to-end experimentation with RL and can be used to systematically investigate the key challenges in developing intelligent control systems. Using the platform, we demonstrate how a 1/18th scale car can learn to drive autonomously using RL with a monocular camera. It is trained in simulation with no additional tuning in physical world and demonstrates: 1) formulation and solution of a robust reinforcement learning algorithm, 2) narrowing the reality gap through joint perception and dynamics, 3) distributed on-demand compute architecture for training optimal policies, and 4) a robust evaluation method to identify when to stop training. It is the first successful large-scale deployment of deep reinforcement learning on a robotic control agent that uses only raw camera images as observations and a model-free learning method to perform robust path planning. We open source our code and video demo on GitHub 2 .

Vision-based reinforcement learning for lane-tracking control

“…Shi et al [4] presented research that involved training reinforcement learning agents in Duckietown, in a similar way to that presented here; however, the focus was mainly on presenting a method that explained the reasoning behind the trained agents rather than the training methods. Also similar to the present study, Balaji et al [5] presented a method for training a road-following policy in a simulator using reinforcement learning and tested the trained agent in the real world, yet their primary contribution is the DeepRacer platform rather than an in-depth analysis of the road-following policy. Almási et al [7] also used reinforcement learning to solve lane following in the Duckietown environment, but their work differs from the present study in the use of an off-policy reinforcement learning algorithm (deep Q-networks (DQNs) [8]); in this study an on-policy algorithm (proximal policy optimization [9]) is used, which achieves significantly better sample efficiency and shorter training times.…”

“…Several methods can be used to constrain and simplify the action space, such as discretisation, clipping some actions or mapping to a lower-dimensional space. Most previous studies [1], [2], [5], [7] have used discrete action spaces, thus the neural network in these policies selected one from a set of hand-crafted actions (steering, throttle combinations), while Kendall et al [3] utilised continuous actions, as has been used in this study.…”

An Open-Source Scale Model Platform for Teaching Autonomous Vehicle Technologies

“…More recently, Amazon has developed a robotics competition called “AWS DeepRacer” [ 14 ]. It includes an 1:18 scale autonomous racing car that is designed to enable thorough evaluation of artificial intelligence navigation models by racing on a real scale track.…”

A Gamified Simulator and Physical Platform for Self-Driving Algorithm Training and Validation

“…Simulation is well-established [1], and small-scale hardware already tests algorithms in lieu of full-scale vehicles [5,6], particularly for algorithms that may be dangerous or costly to test on a full-scale physical platform, such as collision avoidance or high-speed and inclement weather operation. Our approach furthers proven techniques to allow the generation of data from unskilled drivers and through the validation of resulting algorithms on lower cost and more widely accessible hardware than is used today.…”