Deep Reinforcement Learning Supervised Autonomous Exploration in Office Environments

Zhu, Delong; Li, Tingguang; Ho, Danny; Wang, Chaoqun; Meng, Max Q.-H.

doi:10.1109/icra.2018.8463213

Cited by 84 publications

(57 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With semantic reasoning, a robot can achieve human-like performance. Moreover, some learning-based methods toward autonomous exploration have been proposed along with the developments in deep learning approaches [17].…”

Section: B Guidance Of Explorationmentioning

confidence: 99%

Efficient Autonomous Robotic Exploration With Semantic Road Map in Indoor Environments

Wang

Zhu

Teng

et al. 2019

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

In this paper, we propose an integrated framework for the autonomous robotic exploration in indoor environments. Specially, we present a hybrid map, named Semantic Road Map (SRM), to represent the topological structure of the explored environment and facilitate decision-making in the exploration. The SRM is built incrementally along with the exploration process. It is a graph structure with collision-free nodes and edges that are generated within the sensor coverage. Moreover, each node has a semantic label and the expected information gain at that location. Based on the concise SRM, we present a novel and effective decision-making model to determine the next-best-target (NBT) during the exploration. The model concerns the semantic information, the information gain, and the path cost to the target location. We use the nodes of SRM to represent the candidate targets, which enables the target evaluation to be performed directly on the SRM. With the SRM, both the information gain of a node and the path cost to the node can be obtained efficiently. Besides, we adopt the cross-entropy method to optimize the path to make it more informative. We conduct experimental studies in both simulated and real-world environments, which demonstrate the effectiveness of the proposed method.

show abstract

Section: B Guidance Of Explorationmentioning

confidence: 99%

Efficient Autonomous Robotic Exploration With Semantic Road Map in Indoor Environments

Wang

Zhu

Teng

et al. 2019

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Visual observation usually exploits the features of CNN as the inputs to the training networks. For example, Zhu et al introduce using CNN to obtain knowledge about the real-time explored environment [8] in a single agent system. The reinforcement learning model targets the robust and optimal actions for each agent.…”

Section: Issues and Challengesmentioning

confidence: 99%

“…Few works have been done in applying deep reinforcement learning robot exploration problems. Recently, Zhu et al [8] use CNN and LSTM for feature extraction and A3C to approximate the model. However, their method is only for a single agent with a known map and still requiring A* algorithm for planning the path to reach the nearest frontier point similar to the frontier-based method.…”

Section: Learning-based Explorationmentioning

confidence: 99%

“…Through the power iteration, the sequence of eigenvectors {x k } k∈N 0 converges to the importance score (e 1 , ..., e n ) of each node. Furthermore, the weight of each node is defined as the following equation: 8) where e i is the importance score of segment i, indicating the centrality of segment i. Intuitively, after the map is segmented, the border region should be explored preemptively comparing to a centre regions, as centre regions are easier to be visited by any agents in subsequent exploration.…”

Section: Weightmentioning

confidence: 99%

See 1 more Smart Citation

Towards coordinated multi-agent exploration problem via segmentation and reinforcement learning

Chen¹

View full text Add to dashboard Cite

viii the unpredictable and dynamic situations well. These methods can be effective in a single robot case, but assigning multiple robots to explore different locations is challenging since individual robots may interfere with others, making the overall tasks less efficient. A learning-based approach is proposed to solve those issues in this thesis. The algorithm is called CNN-based Multi-agent Proximal Policy Optimization (CMAPPO), which is for allocating multiple robots to explore different environments while over time improving their strategies to allocate the tasks more efficiently and flexibly. This algorithm combines CNN to process multi-channel visual inputs from the observed environment, curriculum learning for improving learning efficiency, and PPO algorithm for motivation based reinforcement learning. Based on the evaluation, the CMAPPO can learn a more efficient strategy for multiple robots (the robot is named agent in the rest of this thesis) to explore the environment than the conventional frontier-based method. This thesis introduces a novel indoor space segmentation-based exploration method which is based on topological clusters of an enclosed environment to perform multiagent exploration. Considering the dynamic situations in the environment, this thesis further develops a new end-to-end deep reinforcement learning architecture for multi-agent exploration strategy by using Convolutional Neural Network (CNN) and Proximal Policy Optimization (PPO).

show abstract

“…Since CNN is quite powerful and has significant generalization ability, the CNN based deep reinforcement learning can achieve and even exceed human level players in playing Atari games and controlling robots, etc. Proposed by Zhu et al, the CNN based reinforcement learning for map exploration has been used for the single agent map exploration prob-lem in relatively simple environment[65]. However, for multi-agent map exploration in a complex environment, the state space for the agents may be very large.…”

mentioning

confidence: 99%

Autonomous multi-agent collaborative environment exploration

Luo¹

View full text Add to dashboard Cite

Deep Reinforcement Learning Supervised Autonomous Exploration in Office Environments

Cited by 84 publications

References 15 publications

Efficient Autonomous Robotic Exploration With Semantic Road Map in Indoor Environments

Efficient Autonomous Robotic Exploration With Semantic Road Map in Indoor Environments

Towards coordinated multi-agent exploration problem via segmentation and reinforcement learning

Autonomous multi-agent collaborative environment exploration

Contact Info

Product

Resources

About