A new noise network and gradient parallelisation‐based asynchronous advantage actor‐critic algorithm

Abstract

Asynchronous advantage actor-critic (A3C) algorithm is a commonly used policy optimization algorithm in reinforcement learning, in which asynchronous is parallel interactive sampling and training, and advantage is a sampling multi-step reward estimation method for computing weights. In order to address the problem of low efficiency and insufficient convergence caused by the traditional heuristic exploration of A3C algorithm in reinforcement learning, an improved A3C algorithm is proposed in this paper. In this algorithm, a noise network function, which updates the noise tensor in an explicit way is constructed to train the agent. Generalised advantage estimation (GAE) is also adopted to describe the dominance function. Finally, a new mean gradient parallelisation method is designed to update the parameters in both the primary and secondary networks by summing and averaging the gradients passed from all the sub-processes to the main process. Simulation experiments were conducted in a gym environment using the PyTorch Agent Net (PTAN) advanced reinforcement learning library, and the results show that the method enables the agent to complete the learning training faster and its convergence during the training process is better. The improved A3C algorithm has a better performance than the original algorithm, which can provide new ideas for subsequent research on reinforcement learning algorithms. K E Y W O R D S asynchronous advantage actor-critic (A3C), generalised advantage estimation (GAE), parallelisation, reinforcement learningThis is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.

Navigation of Autonomous Vehicles using Reinforcement Learning with Generalized Advantage Estimation

“…To address this problem, a reinforcement learning approach using Generalized Advantage Estimation (GAE) to enable autonomous vehicles to learn to navigate complex environments is proposed [20], [21]. Reinforcement learning enables agents to learn by interacting with their environment and receiving feedback as rewards or penalties [22].…”

Deep feature fusion‐based stacked denoising autoencoder for tag recommendation systems

“…To facilitate the management and retrieval of available information, tag recommendation systems have become an indispensable component of many commerce platforms [1], enabling users to more quickly select products of interest. A large number of applications are gradually gaining attention under the continuous development and popularity of intelligent technology [2, 3]. Many web applications employ tag recommendation systems to help users index resources of interest, such as Flickr, Delicious, LastFm, MovieLens and Bibsonomy.…”

“…attention under the continuous development and popularity of intelligent technology [2,3]. Many web applications employ tag recommendation systems to help users index resources of interest, such as Flickr, Delicious, LastFm, MovieLens and Bibsonomy.…”