Target‐driven visual navigation in indoor scenes using reinforcement learning and imitation learning

Abstract

Here, the challenges of sample efficiency and navigation performance in deep reinforcement learning for visual navigation are focused and a deep imitation reinforcement learning approach is proposed. Our contributions are mainly three folds: first, a framework combining imitation learning with deep reinforcement learning is presented, which enables a robot to learn a stable navigation policy faster in the target-driven navigation task. Second, the surrounding images is taken as the observation instead of sequential images, which can improve the navigation performance for more information. Moreover, a simple yet efficient template matching method is adopted to determine the stop action, making the system more practical. Simulation experiments in the AI-THOR environment show that the proposed approach outperforms previous end-to-end deep reinforcement learning approaches, which demonstrate the effectiveness and efficiency of our approach.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Search for or Navigate to? Dual Adaptive Thinking for Object Navigation

“…Following the previous works (Mirowski et al 2017;Fang et al 2021), we treat this task as a reinforcement learning problem and utilize the asynchronous advantage actor-critic (A3C) algorithm (Mnih et al 2016). However, in the search thinking network, the complex multi-head attention calculations are difficult to directly learn by the reinforcement learning (Du, Yu, and Zheng 2021); thus, we use imitation learning to pretrain the search thinking network.…”

Unbiased Directed Object Attention Graph for Object Navigation

“…Previous works [36,38] [7,20], we treat this task as a reinforcement learning problem and utilize the asynchronous advantage actor-critic (A3C) algorithm [22], which applies policy gradients to assist the agent in choosing an appropriate action 𝑎 𝑡 in the high-dimensional action space 𝐴. In accordance with the done reminder operation presented in [41], when the agent detects the target, we use the target detection confidence to explicitly enhance the probability of the 𝐷𝑜𝑛𝑒 action in the action domain 𝐴 𝑡 ∈ R 1×6 .…”

Maximum Entropy Dueling Network Architecture in Atari Domain

“…the well known function approximation ability of deep architectures made them a great choice for regressing various RL functions (that we discuss in the next section), one of the earliest implementation is TD-Gammon, neural network that reached champion-level performance in Backgammon decades ago [16]. Current methods, address the highly complex domain of inputs such as images and videos [2,17,18,19,20]. One of the key components of RL is Q-function, van Hasselt showed that the single estimator in Q-learning is suffering from over-estimation, hence a new algorithm with double estimator was proposed that improved the learning process immensely [21].…”