Autonomous Navigation of Robots: Optimization with DQN

Escobar-Naranjo, Juan; Caiza, Gustavo; Ayala, Paulina; Jordán, Edisson P.; García, Carlos A.; García, Marcelo V.

doi:10.3390/app13127202

Cited by 9 publications

(4 citation statements)

References 96 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, while excelling in radioactive grid scenarios, it might lack adaptability across varied complex environments and might not prioritize real-time obstacle avoidance. In [ 33 ], a novel algorithm integrating reinforcement learning with the Deep Q-Network (DQN) aimed to empower agents for real-time decision making and trajectory planning in Gazebo simulations. However, this approach, while promoting exploration, might overlook real-time obstacle avoidance during trajectory planning, potentially limiting its effectiveness in dynamic environments.…”

Section: Related Workmentioning

confidence: 99%

Enhancing Stability and Performance in Mobile Robot Path Planning with PMR-Dueling DQN Algorithm

Deguale,

Yu,

Sinishaw

et al. 2024

Sensors

View full text Add to dashboard Cite

Path planning for mobile robots in complex circumstances is still a challenging issue. This work introduces an improved deep reinforcement learning strategy for robot navigation that combines dueling architecture, Prioritized Experience Replay, and shaped Rewards. In a grid world and two Gazebo simulation environments with static and dynamic obstacles, the Dueling Deep Q-Network with Modified Rewards and Prioritized Experience Replay (PMR-Dueling DQN) algorithm is compared against Q-learning, DQN, and DDQN in terms of path optimality, collision avoidance, and learning speed. To encourage the best routes, the shaped Reward function takes into account target direction, obstacle avoidance, and distance. Prioritized replay concentrates training on important events while a dueling architecture separates value and advantage learning. The results show that the PMR-Dueling DQN has greatly increased convergence speed, stability, and overall performance across conditions. In both grid world and Gazebo environments the PMR-Dueling DQN achieved higher cumulative rewards. The combination of deep reinforcement learning with reward design, network architecture, and experience replay enables the PMR-Dueling DQN to surpass traditional approaches for robot path planning in complex environments.

show abstract

Section: Related Workmentioning

confidence: 99%

Enhancing Stability and Performance in Mobile Robot Path Planning with PMR-Dueling DQN Algorithm

Deguale,

Yu,

Sinishaw

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…The DQN model offers adaptive decision-making capabilities, learning from experience and adjusting its decision-making process based on real-world performance [42]. In our research, we implement the DQN to optimize the threshold time for releasing the DL model in the video surveillance system, as shown in Figure 3.…”

Section: Dqn-based Controlling Threshold Modulementioning

confidence: 99%

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework

Bazarov Ravshan Ugli,

Mohammed,

et al. 2024

Sensors

View full text Add to dashboard Cite

Video surveillance systems are integral to bolstering safety and security across multiple settings. With the advent of deep learning (DL), a specialization within machine learning (ML), these systems have been significantly augmented to facilitate DL-based video surveillance services with notable precision. Nevertheless, DL-based video surveillance services, which necessitate the tracking of object movement and motion tracking (e.g., to identify unusual object behaviors), can demand a significant portion of computational and memory resources. This includes utilizing GPU computing power for model inference and allocating GPU memory for model loading. To tackle the computational demands inherent in DL-based video surveillance, this study introduces a novel video surveillance management system designed to optimize operational efficiency. At its core, the system is built on a two-tiered edge computing architecture (i.e., client and server through socket transmission). In this architecture, the primary edge (i.e., client side) handles the initial processing tasks, such as object detection, and is connected via a Universal Serial Bus (USB) cable to the Closed-Circuit Television (CCTV) camera, directly at the source of the video feed. This immediate processing reduces the latency of data transfer by detecting objects in real time. Meanwhile, the secondary edge (i.e., server side) plays a vital role by hosting a dynamically controlling threshold module targeted at releasing DL-based models, reducing needless GPU usage. This module is a novel addition that dynamically adjusts the threshold time value required to release DL models. By dynamically optimizing this threshold, the system can effectively manage GPU usage, ensuring resources are allocated efficiently. Moreover, we utilize federated learning (FL) to streamline the training of a Long Short-Term Memory (LSTM) network for predicting imminent object appearances by amalgamating data from diverse camera sources while ensuring data privacy and optimized resource allocation. Furthermore, in contrast to the static threshold values or moving average techniques used in previous approaches for the controlling threshold module, we employ a Deep Q-Network (DQN) methodology to manage threshold values dynamically. This approach efficiently balances the trade-off between GPU memory conservation and the reloading latency of the DL model, which is enabled by incorporating LSTM-derived predictions as inputs to determine the optimal timing for releasing the DL model. The results highlight the potential of our approach to significantly improve the efficiency and effective usage of computational resources in video surveillance systems, opening the door to enhanced security in various domains.

show abstract

“…Traditional global path planning approaches, such as Dijkstra's algorithms [19], heavily rely on precomputed maps and face limitations in adapting to dynamic changes. In contrast, local path planning methods, like the Velocity Obstacle method and the Dynamic Window Approach, focus on real-time obstacle avoidance, considering the robot's immediate surroundings [20,21]. While suitable for reactive navigation, these methods may lack the ability to plan globally.…”

Section: Related Workmentioning

confidence: 99%

Enhancing Mobile Robot Navigation: Optimization of Trajectories through Machine Learning Techniques for Improved Path Planning Efficiency

Al-Kamil,

Szabolcsi

2024

Mathematics

View full text Add to dashboard Cite

Efficient navigation is crucial for intelligent mobile robots in complex environments. This paper introduces an innovative approach that seamlessly integrates advanced machine learning techniques to enhance mobile robot communication and path planning efficiency. Our method combines supervised and unsupervised learning, utilizing spline interpolation to generate smooth paths with minimal directional changes. Experimental validation with a differential drive mobile robot demonstrates exceptional trajectory control efficiency. We also explore Motion Planning Networks (MPNets), a neural planner that processes raw point-cloud data from depth sensors. Our tests demonstrate MPNet’s ability to create optimal paths using the Probabilistic Roadmap (PRM) method. We highlight the importance of correctly setting parameters for reliable path planning with MPNet and evaluate the algorithm on various path types. Our experiments confirm that the trajectory control algorithm works effectively, consistently providing precise and efficient trajectory control for the robot.

show abstract

Autonomous Navigation of Robots: Optimization with DQN

Cited by 9 publications

References 96 publications

Enhancing Stability and Performance in Mobile Robot Path Planning with PMR-Dueling DQN Algorithm

Enhancing Stability and Performance in Mobile Robot Path Planning with PMR-Dueling DQN Algorithm

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework

Enhancing Mobile Robot Navigation: Optimization of Trajectories through Machine Learning Techniques for Improved Path Planning Efficiency

Contact Info

Product

Resources

About