Deep Reinforcement Learning-Based 2.5D Multi-Objective Path Planning for Ground Vehicles: Considering Distance and Energy Consumption

Wu, Xiru; Huang, Shuqiao; Huang, Guoming

doi:10.3390/electronics12183840

Cited by 2 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Q value of the target network is calculated differently from the DQN-PER-TEAE algorithm. Specifically, in the DDQN-PER-TEAE algorithm, the rule for updating the Q value of the target network is: if episode terminates at time step t + 1, then update according to Equation (19), otherwise update according to Equation (20). By decoupling the action selection and the calculation of the target Q value, the DDQN-PER-TEAE algorithm can reduce the overestimation bias in the DQN-PER-TEAE algorithm.…”

Section: ) Ddqn-per-teaementioning

confidence: 99%

“…The pseudocode of the DDQN-PER-TEAE algorithm is similar to the DQN-PER-TEAE algorithm, except for changes to the network parameters and update formulas. In DDQN-PER-TEAE, the current and target network parameters are represented by θ and θ , and the update formulas in lines 10 and 16 are changed to Equations ( 21), ( 19) and (20), respectively. Compared to the DDQN-PER algorithm, DDQN-PER-TEAE avoids over-estimation and poor model robustness, and allows the agent to explore or exploit actions based on their environmental knowledge.…”

Section: ) Ddqn-per-teaementioning

confidence: 99%

“…Tasks with Markovian properties, such as scheduling unmanned drones for emergency power supply transport or formulating distribution strategies for warehouse locations in the supply chain, can all be addressed using the RLMDP method [16,17]. RLMDP primarily relies on learning from empirical knowledge, allowing decision makers to adapt their strategies over time to maximize their payoffs [18][19][20]. However, it is crucial to acknowledge that incorporating learning into decision making introduces additional complexities [21,22].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Reinforcement Learning Method of Solving Markov Decision Processes: An Adaptive Exploration Model Based on Temporal Difference Error

Wang,

Yang,

Chen

et al. 2023

Electronics

View full text Add to dashboard Cite

Traditional backward recursion methods face a fundamental challenge in solving Markov Decision Processes (MDP), where there exists a contradiction between the need for knowledge of optimal expected payoffs and the inability to acquire such knowledge during the decision-making process. To address this challenge and strike a reasonable balance between exploration and exploitation in the decision process, this paper proposes a novel model known as Temporal Error-based Adaptive Exploration (TEAE). Leveraging reinforcement learning techniques, TEAE overcomes the limitations of traditional MDP solving methods. TEAE exhibits dynamic adjustment of exploration probabilities based on the agent’s performance, on the one hand. On the other hand, TEAE approximates the optimal expected payoff function for subprocesses after specific states and times by integrating deep convolutional neural networks to minimize the temporal difference error between the dual networks. Furthermore, the paper extends TEAE to DQN-PER and DDQN-PER methods, resulting in DQN-PER-TEAE and DDQN-PER-TEAE variants, which not only demonstrate the generality and compatibility of the TEAE model with existing reinforcement learning techniques but also validate the practicality and applicability of the proposed approach in a broader MDP reinforcement learning context. To further validate the effectiveness of TEAE, the paper conducts a comprehensive evaluation using multiple metrics, compares its performance with other MDP reinforcement learning methods, and conducts case studies. Ultimately, simulation results and case analyses consistently indicate that TEAE exhibits higher efficiency, highlighting its potential in driving advancements in the field.

show abstract

Section: ) Ddqn-per-teaementioning

confidence: 99%

Section: ) Ddqn-per-teaementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Reinforcement Learning Method of Solving Markov Decision Processes: An Adaptive Exploration Model Based on Temporal Difference Error

Wang,

Yang,

Chen

et al. 2023

Electronics

View full text Add to dashboard Cite

show abstract

“…These vehicles often navigate complex off-road terrains, where energy consumption poses limitations on their operational range and duration. Researchers have delved into research on energy storage technology, electrical machine control techniques, and internal combustion engine advancements to enhance energy efficiency in this context [18].…”

Section: Introductionmentioning

confidence: 99%

Enhanced Energy Efficiency through Path Planning for Off-Road Missions of Unmanned Tracked Electric Vehicle

İnal,

Cansever,

Yalçın

et al. 2024

Vehicles

View full text Add to dashboard Cite

The primary objective of this research is to address the existing gap about the use of a path-planning algorithm that will reduce energy consumption in off-road applications of tracked electric vehicles. The study focuses on examining various off-road terrains and their impact on energy consumption to validate the effectiveness of the proposed solution. To achieve this, a tracked electric vehicle energy model that incorporates vehicle dynamics is developed and verified using real vehicle driving data logs. This model serves as the foundation for devising a strategy that can effectively enhance the energy efficiency of off-road tracked electric vehicles in real-world scenarios. The analysis involves a thorough examination of different off-road terrains to identify strategies that can adapt to diverse landscapes. The path planning strategy employed in this study is a modified version of the A*, called the Energy-Efficient Path Planning (EEPP) algorithm, specifically tailored for the dynamic energy consumption model of off-road tracked electric vehicles. The energy consumption of the produced paths is then compared using the validated energy consumption model of the tracked electric vehicle. It is important to note that the identification of an energy-efficient path heavily relies on the characteristics of the vehicle and the dynamic energy consumption model that has been developed. Furthermore, the algorithm takes into account real-world and practical considerations associated with off-road applications during its development and evaluation process. The results of the comprehensive analysis comparing the EEPP algorithm with the A* algorithm demonstrate that our proposed approach achieves energy savings of up to 6.93% and extends the vehicle’s operational range by 7.45%.

show abstract

Deep Reinforcement Learning-Based 2.5D Multi-Objective Path Planning for Ground Vehicles: Considering Distance and Energy Consumption

Cited by 2 publications

References 34 publications

A Reinforcement Learning Method of Solving Markov Decision Processes: An Adaptive Exploration Model Based on Temporal Difference Error

A Reinforcement Learning Method of Solving Markov Decision Processes: An Adaptive Exploration Model Based on Temporal Difference Error

Enhanced Energy Efficiency through Path Planning for Off-Road Missions of Unmanned Tracked Electric Vehicle

Contact Info

Product

Resources

About