Double Critic Deep Reinforcement Learning for Mapless 3D Navigation of Unmanned Aerial Vehicles

Grando, Ricardo Bedin; Jesus, Junior Costa de; Kich, Victor Augusto; Kolling, Alisson Henrique; Drews-Jr, Paulo L. J.

doi:10.1007/s10846-021-01568-y

Cited by 29 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This information allows the estimation of the agent's degree of learning in the environment, as the improved in terms of reward is intrinsically linked to the agent's performance in the environment it must navigate. All environments used for training the network are provided by the Grando et al [29]. However, some changes are made in the source code of the Gazebo simulation to use the camera of the simulated UAV.…”

Section: Resultsmentioning

confidence: 99%

“…The study around mapless navigation is extensively explored using terrestrial mobile robots [3]. Autonomous navigation of aerial mobile robots using Deep-RL approaches is less frequent and mainly focused on approaches that avoid the use of visual information [6], [10] or using simplified information, without contrastive learning [7]- [9], [13].…”

Section: Related Workmentioning

confidence: 99%

“…The algorithms are developed using the PyTorch library and OpenCV [28]. The robotic UAV platform used in the simulation is designed by Grando et al [29]. The vehicle is based on a real vehicle called Hydrone [30], [31].…”

Section: B Experimental Setupmentioning

confidence: 99%

“…methods are capable of providing an action response for continuous systems [4], [5], as well as the Deep-RL control of UAVs in simulated environments [6]. It is possible to see the application of these techniques for the mapless navigation of UAVs in models of unknown dynamics in many environments [6]- [10]. However, one of the significant challenges of Deep-RL techniques is the limitation given by the immense need for labeled data which is difficult to collect in the real world [11].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Jesus¹,

Kich²,

Kolling³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reinforcement Learning (RL) has presented an impressive performance in video games through raw pixel imaging and continuous control tasks. However, RL performs poorly with high-dimensional observations such as raw pixel images. It is generally accepted that physical state-based RL policies such as laser sensor measurements give a more sampleefficient result than learning by pixels. This work presents a new approach that extracts information from a depth map estimation to teach an RL agent to perform the mapless navigation of Unmanned Aerial Vehicle (UAV). We propose the Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning(Depth-CUPRL) that estimates the depth of images with a prioritized replay memory. We used a combination of RL and Contrastive Learning to lead with the problem of RL based on images. From the analysis of the results with Unmanned Aerial Vehicles (UAVs), it is possible to conclude that our Depth-CUPRL approach is effective for the decision-making and outperforms state-of-the-art pixel-based approaches in the mapless navigation capability.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: B Experimental Setupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Jesus¹,

Kich²,

Kolling³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Mapless planners [3][4][5]7] only employ a local map that is just enough for local planning, and a memoryless one [6] plans directly on sensor data. Distinguishing from those, map-based planning systems [8,9] generally integrate global maps [10,11] a priori and a global planning algorithm such as RRT* [12] to guarantee planning completeness in tasks such as exploration, [13][14][15][16][17] or navigating toward a goal [18][19][20]. All mapless and memoryless planners cannot access any map but use sensor data.…”

Section: Introductionmentioning

confidence: 99%

Depth-based Sampling and Steering Constraints for Memoryless Local Planners

Nguyen¹,

Nguyen²,

Choudhury³

et al. 2022

Preprint

View full text Add to dashboard Cite

By utilizing only depth information, the paper introduces a novel but efficient local planning approach that enhances not only computational efficiency but also planning performances for memoryless local planners. The sampling is first proposed to be based on the depth data which can identify and eliminate a specific type of in-collision trajectories in the sampled motion primitive library. More specifically, all the obscured primitives' endpoints are found through querying the depth values and excluded from the sampled set, which can significantly reduce the computational workload required in collision checking. On the other hand, we furthermore propose a steering mechanism also based on the depth information to effectively prevent an autonomous vehicle from getting stuck when facing a large convex obstacle, providing a higher level of autonomy for a planning system. Our steering technique is theoretically proved to be complete in scenarios of convex obstacles. To evaluate effectiveness of the proposed DEpth based both Sampling and Steering (DESS) methods, we implemented them in the synthetic environments where a quadrotor

show abstract

Double Deep Reinforcement Learning Techniques for Low Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots

Dotto

Kich

Kolling

et al. 2023

Intelligent Systems Design and Applications

View full text Add to dashboard Cite

Double Critic Deep Reinforcement Learning for Mapless 3D Navigation of Unmanned Aerial Vehicles

Cited by 29 publications

References 23 publications

Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Depth-based Sampling and Steering Constraints for Memoryless Local Planners

Double Deep Reinforcement Learning Techniques for Low Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots

Contact Info

Product

Resources

About