Efficient Exploration in Constrained Environments with Goal-Oriented Reference Path

Ota, Kei; Sasaki, Yoko; Jha, Devesh K.; Yoshiyasu, Yusuke; Kanezaki, Asako

doi:10.1109/iros45743.2020.9341620

Cited by 22 publications

(15 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, [15] uses reinforcement learning (RL) to acquire local policy for steering within SMP methods. Similarly, [16] [17] constructs high-level landmarks, which are later connected by local RL policies. However, these methods inherit RL limitations such as requiring exhaustive interactions with their environments for learning.…”

Section: Related Workmentioning

confidence: 99%

MPC-MPNet: Model-Predictive Motion Planning Networks for Fast, Near-Optimal Planning under Kinodynamic Constraints

Miao

Qureshi

et al. 2021

Preprint

View full text Add to dashboard Cite

Kinodynamic Motion Planning (KMP) is to find a robot motion subject to concurrent kinematics and dynamics constraints. To date, quite a few methods solve KMP problems and those that exist struggle to find near-optimal solutions and exhibit high computational complexity as the planning space dimensionality increases. To address these challenges, we present a scalable, imitation learning-based, Model-Predictive Motion Planning Networks framework that quickly finds near-optimal path solutions with worst-case theoretical guarantees under kinodynamic constraints for practical underactuated systems. Our framework introduces two algorithms built on a neural generator, discriminator, and a parallelizable Model Predictive Controller (MPC). The generator outputs various informed states towards the given target, and the discriminator selects the best possible subset from them for the extension. The MPC locally connects the selected informed states while satisfying the given constraints leading to feasible, near-optimal solutions. We evaluate our algorithms on a range of cluttered, kinodynamically constrained, and underactuated planning problems with results indicating significant improvements in computation times, path qualities, and success rates over existing methods.

show abstract

Section: Related Workmentioning

confidence: 99%

MPC-MPNet: Model-Predictive Motion Planning Networks for Fast, Near-Optimal Planning under Kinodynamic Constraints

Miao

Qureshi

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…It also performed in a scenario with multiple humans simulating a crowd. Ota et al [20] developed an approch based on the SAC technique to efficiently navigate. It was also done in simulation, with the agent learning the course.…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Grando

Jesus

Drews-Jr

2020

2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Educ

View full text Add to dashboard Cite

This paper presents a novel deep reinforcement learning-based system for 3D mapless navigation for Unmanned Aerial Vehicles (UAVs). Instead of using a image-based sensing approach, we propose a simple learning system that uses only a few sparse range data from a distance sensor to train a learning agent. We based our approaches on two state-of-art double critic Deep-RL models: Twin Delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC). We show that our two approaches manage to outperform an approach based on the Deep Deterministic Policy Gradient (DDPG) technique and the BUG2 algorithm. Also, our new Deep-RL structure based on Recurrent Neural Networks (RNNs) outperforms the current structure used to perform mapless navigation of mobile robots. Overall, we conclude that Deep-RL approaches based on double critic with Recurrent Neural Networks (RNNs) are better suited to perform mapless navigation and obstacle avoidance of UAVs.

show abstract

“…In our proposed work, this is achieved as the path encodes the goal and automatically tries to avoid obstacles in the environment, too. The closest work to ours is [12,18]. [12] learns an RL agent that optimizes trajectory for a 6-DoF manipulator arm.…”

Section: Related Workmentioning

confidence: 99%

“…Our approach, however, can deal with changes in the environment by conditioning an RL agent with a reference path which is also generated based on an observation of the current environment. [18] considers a similar setting for 2-dimensional robot navigation tasks only in simulation. Our method, however, considers higher-dimensional path planning, and is evaluated on real systems as well as simulated ones.…”

Section: Related Workmentioning

confidence: 99%

Deep Reactive Planning in Dynamic Environments

Ota¹,

Jha²,

Onishi³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

4 AIST Figure 1: Our proposed agent learns an end-to-end reactive planning technique by combining traditional path planning algorithms, supervised learning (SL) and reinforcement learning (RL) algorithms in a synergistic way. A deep CNN is used to learn the sequence of waypoints obtained from a kinematic planning algorithm (e.g., a Bidirectional RRT*) given a depth image of the environment. The agent learns to follow arbitrary waypoints using path-conditioned RL, thus resulting in efficient exploration. We show that our trained agent can achieve good sample efficiency, as well as generalization to novel environments in simulation as well as real environments. The whole learning process is done in the simulator by learning a Real2Sim transfer function to make the training process efficient and suitable for robotic systems.

show abstract

Efficient Exploration in Constrained Environments with Goal-Oriented Reference Path

Cited by 22 publications

References 24 publications

MPC-MPNet: Model-Predictive Motion Planning Networks for Fast, Near-Optimal Planning under Kinodynamic Constraints

MPC-MPNet: Model-Predictive Motion Planning Networks for Fast, Near-Optimal Planning under Kinodynamic Constraints

Deep Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Deep Reactive Planning in Dynamic Environments

Contact Info

Product

Resources

About