Orthogonal Policy Gradient and Autonomous Driving Application

Abstract

One less addressed issue of deep reinforcement learning is the lack of generalization capability based on new state and new target, for complex tasks, it is necessary to give the correct strategy and evaluate all possible actions for current state. Fortunately, deep reinforcement learning has enabled enormous progress in both subproblems: giving the correct strategy and evaluating all actions based on the state.In this paper we present an approach called orthogonal policy gradient descent(OPGD) that can make agent learn the policy gradient based on the current state and the actions set, by which the agent can learn a policy network with generalization capability. we evaluate the proposed method on the 3D autonomous driving enviroment TORCS compared with the baseline model, detailed analyses of experimental results and proofs are also given.

Synthetic Data for Deep Learning

“…We do not aim to provide a full in-depth survey of the entire field and only note that despite its long history TORCS is being actively used for research purposes up to this day. In particular, Sallab et al [516,517] use it in their deep reinforcement learning frameworks for lane keeping assist and autonomous driving, Xiong et al [661] add safety-based control on top of deep RL, Wang et al [626] train a deep RL agent for autonomous driving in TORCS, Barati et al [37] use it to add multi-view inputs for deep RL agents, Li et al [352] develop Visual TORCS, a deep RL environment based on TORCS, Ando, Lubashevsky et al [20,381] use TORCS to study the statistical properties of human driving, Glassner et al [202] shift the emphasis to trajectory learning, Luo et al [383] use TORCS as the main test environment for a new variation of the policy gradient algorithm, Liu et al [369] make use of the multimodal sensors available in TORCS for end-to-end learning, Xu et al [576] train a segmentation network and feed segmentation results to the RL agent in order to unify synthetic imagery from TORCS and real data, and so on. In an interesting recent work, Choi et al [114] consider the driving experience transfer problem but consider a transfer not from a synthetic simulator to the real domain but from one simulator (TORCS) to another (GTA V).…”

Lane Following Method Based on Improved DDPG Algorithm

“…In recent years, artificial intelligence technology applied to autonomous driving has developed rapidly, especially reinforcement learning technology [ 4 , 5 , 6 , 7 , 8 ]. The first control example based on reinforcement learning (RL) was inspired by the concept of ALVINN [ 9 ].…”