Adversarial Imitation Learning via Random Search

Abstract

Developing agents that can perform challenging complex tasks is the goal of reinforcement learning. The modelfree reinforcement learning has been considered as a feasible solution. However, the state of the art research has been to develop increasingly complicated techniques. This increasing complexity makes the reconstruction difficult. Furthermore, the problem of reward dependency is still exists. As a result, research on imitation learning, which learns policy from a demonstration of experts, has begun to attract attention. Imitation learning directly learns policy based on data on the behavior of the experts without the explicit reward signal provided by the environment. However, imitation learning tries to optimize policies based on deep reinforcement learning such as trust region policy optimization. As a result, deep reinforcement learning based imitation learning also poses a crisis of reproducibility. The issue of complex model-free model has received considerable critical attention. A derivative-free optimization based reinforcement learning and the simplification on policies obtain competitive performance on the dynamic complex tasks. The simplified policies and derivative free methods make algorithm be simple. The reconfiguration of research demo becomes easy. In this paper, we propose an imitation learning method that takes advantage of the derivative-free optimization with simple linear policies. The proposed method performs simple random search in the parameter space of policies and shows computational efficiency. Experiments in this paper show that the proposed model, without a direct reward signal from the environment, obtains competitive performance on the MuJoCo locomotion tasks.

SwapTransformer: Highway Overtaking Tactical Planner Model via Imitation Learning on OSHA Dataset

“…Imitation learning is one of the earliest and most successful methods for Autonomous Driving tasks. Authors in [16], [17], and [18] have used different variations of Imitation Learning to solve overtaking or lane changing specifically.…”

Adaptive Real-Time Offloading Decision-Making for Mobile Edges: Deep Reinforcement Learning Framework and Simulation Results

“…In addition, there are a number of control-related studies for safe autonomous driving in highway environments [19,20]. Shin et al proposed a novel inverse-reinforcement and imitation learning algorithm, which is one of major reinforcement learning algorithms nowadays, for autonomous driving control under the benefits of augmented random search [21,22]. In addition, there exists RL applications to game playing.…”

“…Finally, the AlphaGo Zero has achieved more than 100 wins against AlphaGo, which won against world-champion human Go players. The algorithms in [18][19][20][21][22][23] conduct engineering for their specific applications well such as autonomous driving and game AI. Therefore, they are all independent of the communications and networks specific key consideration factors such as energy and delay optimization.…”

Parallelized and randomized adversarial imitation learning for safety-critical self-driving vehicles

“…These problems introduce the difficulties in training robust and safety-critical autonomous self-driving policies; the trained policies have not been successfully deployed to autonomous vehicles yet [30], [31]. Recently, augmented random search (ARS) which is based on the natural gradient policy is proposed [32], [33]. According to the fact that the ARS is based on derivative-free simple linear policy optimization, it is relatively easy to reconfigure the robust trained policy that works with reasonably acceptable performance.…”