BND*-DDQN: Learn to Steer Autonomously Through Deep Reinforcement Learning

Abstract

HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Multimodal Deep Reinforcement Learning with Auxiliary Task for Obstacle Avoidance of Indoor Mobile Robot

“…Apart from average accumulated reward, we also compare the success rate of each method in this work. By following previous work [ 13 ], an episode is considered to be successful if no collision occurs within 300 steps and the success rate represents the ratio of successful episodes in 50 episodes. In order to assess the generalization capability, we tested all methods in three different kinds of virtual environments directly without any fine-tuning.…”

“…At the same time, the next state is passed through the online network to determine optimal action and also fed into the target network to compute target Q-value of the determined action by using Equation (10). Based on the Qvalues obtained by online network and target network, we define the loss function of DRL as below: (13) In addition, the online network is also used for the proposed auxiliary task by using Equation (12). Therefore, the parameters of the multimodal representation in the online network are updated simultaneously by two-part back-propagation error and two Adam optimizers [32] are used with a same learning rate of 0.0001.…”

“…Despite these 2D laser-only methods showing promising results, the 2D laser range finder may not be sufficient to describe the surrounding environment due to its limited vertical field of view [ 11 ], given the fact that the 2D laser range finder cannot detect obstacles lying below or above its line of scanning [ 12 ]. In contrast, depth image provides more environment information than the 2D laser range finder does [ 13 ], and therefore many researchers attempt to use depth images for obstacle avoidance [ 13 , 14 , 15 ]. For example, Tai et al use a 2D CNN to directly process raw depth images for learning efficient features for obstacle avoidance [ 14 ].…”

“…For example, Tai et al use a 2D CNN to directly process raw depth images for learning efficient features for obstacle avoidance [ 14 ]. Similarly, in a DRL-based approach, Wu et al use stacked depth images and difference images between successive frames as input and utilize a two-stream CNN to process two sets of data, then the extracted features are concatenated and mapped to angular and linear velocities commands for a mobile robot [ 13 ]. However, depth image has a limited horizontal field of view that may affect the performance of obstacle avoidance in crowd environment [ 16 ].…”

Learning to Navigate in a VUCA Environment: Hierarchical Multi-expert Approach

“…Recently, deep reinforcement learning (DRL) [16] has been employed to settle navigation problem in an end-to-end manner and achieved considerable successes [17], [18], [19]. Tai et al [20] trained a DQN agent for obstacle avoidance in a simulated indoor environment.…”