An Optimized Path Planning Method for Container Ships in Bohai Bay Based on Improved Deep Q-Learning

Gao, Xuanyu; Dong, Yitao; Han, Yi

doi:10.1109/access.2023.3307480

Cited by 4 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Q-learning was introduced to learn the action reward model for judging ship autonomous collision avoidance decisions. Gao et al [41] proposed an improved deep Q-Learning algorithm, which is used to solve the limitations of the DQN algorithm in adaptability and low success rate in ship path planning.…”

Section: Collision Avoidancementioning

confidence: 99%

Ship collision avoidance decision-making research in coastal waters considering uncertainty of target ships

Gao,

Zhang

2024

Brodogradnja

View full text Add to dashboard Cite

Ship collision avoidance has always been a concern and it is crucial for achieving safe navigation of ships at sea. There are many studies on ship collision avoidance in open water, but less attention on coastal waters considering the uncertainty of target ships due to the complexity of the environment and traffic flow. In this paper, collision avoidance decision-making research in coastal waters considering the uncertainty of target ships was proposed. Firstly, accurate ship trajectories are obtained by preprocessing the raw Automatic Identification System (AIS) data. Subsequently, the processed trajectories are clustered using the Ordering Points to Identify the Clustering Structure (OPTICS) algorithm and Hausdorff distance, acquiring a dataset for trajectory prediction of target ships. Then, a mixed Gaussian model is utilized to calculate the prior probability distribution of the prediction model, thus establishing a trajectory prediction model that considers the uncertainty of the target ship. Finally, ship maneuverability is simulated using the Mathematical Model Group (MMG) and Proportion Integration Differentiation (PID) models, and a collision avoidance decision-making model for ships is constructed. The proposed algorithm has been tested and verified in a case study. The results show that the approach effectively predicts the trajectory of the target ship and facilitates informed collision avoidance decision-making.

show abstract

Section: Collision Avoidancementioning

confidence: 99%

Ship collision avoidance decision-making research in coastal waters considering uncertainty of target ships

Gao,

Zhang

2024

Brodogradnja

View full text Add to dashboard Cite

show abstract

“…Gao et al [26] addressed the slow convergence issue in the DQN algorithm combined with prioritized experience replay by introducing the k-means algorithm to handle the agent's state space. This improvement enhanced the convergence speed of the DQN algorithm.…”

Section: A Algorithm Based On Dqnmentioning

confidence: 99%

Path Planning for Outdoor Mobile Robots Based on IDDQN

Shuhai,

Shangjie,

Cun

2024

IEEE Access

View full text Add to dashboard Cite

Path planning is one of the research hotspots for outdoor mobile robots. This paper addresses the issues of slow convergence and low accuracy in the Double Deep Q Network (DDQN) method in environments with many obstacles in the context of deep reinforcement learning. A new algorithm, Improve Double Deep Q Network (IDDQN), is proposed, which utilizes second-order temporal difference methods and a binary tree data structure to improve the DDQN method. The improved method evaluates the actions of the current robot using second-order temporal difference methods and employs a binary tree structure to store the results obtained from these methods, replacing the traditional experience pool structure. The environment is constructed using a grid method, programmed in the Python language, with two twodimensional grid maps created for simple and complex environments. DDQN and four related deep reinforcement learning methods, such as Multi-step updates and Experience Classification Double Deep Q Network (ECMS-DDQN), are compared through simulation experiments with the IDDQN method. Simulation results indicate that the IDDQN method improves various path planning metrics compared to the DDQN method and other relevant reinforcement learning methods. In the simple environment, IDDQN method exhibits a 26.89% improvement in step convergence time, a 22.58% improvement in reward convergence time, and a 10.30% improvement in average reward value after convergence compared to the original DDQN algorithm. It also outperforms other simulated methods in the simple environment, although the difference is not significant. In the complex environment, the IDDQN method avoids falling into local optima compared to other methods, demonstrating the accuracy of its strategy in complex environments. Other methods show artificially high average reward values after converging in local optima, lacking reference value. In the complex environment, IDDQN method exhibits a 33.22% improvement in step convergence time and a 25.47% improvement in reward convergence time compared to the original DDQN algorithm, clearly surpassing other participating simulated methods. The data above indicate that the IDDQN method improves both convergence speed and accuracy compared to the DDQN method and the relevant improvement methods simulated in this paper. Particularly in environments with many obstacles, the performance improvement is evident, allowing for effective path planning in such environments.

show abstract

“…The big data mining method is based on the statistical analysis of a large number of ship route data from the automatic identification system (AIS) to improve shipping efficiency [15][16][17][18][19]. This method usually considers historical ship trajectory data, using clustering methods for statistical analysis and establishing a route trajectory model to improve the safety of navigation.…”

Section: Introductionmentioning

confidence: 99%

Ship Global Traveling Path Optimization via a Novel Non-Dominated Sorting Genetic Algorithm

Zhao,

Zhao

2024

JMSE

View full text Add to dashboard Cite

Due to the intensification of economic globalization and the impact of global warming, the development of methods to reduce shipping costs and reduce carbon emissions has become crucial. In this study, a multi-objective optimization algorithm was designed to plan the optimal ship route for safe cross-ocean navigation under complex sea conditions. Based on the traditional non-dominated sorting genetic algorithm, considering ship stability and complex marine environment interference, a non-dominated sorting genetic algorithm model considering energy consumption was designed with the energy consumption and navigation time of the ship as the optimization objectives. The experimental results show that although the proposed method is 101.23 nautical miles more than the large ring route, and the voyage is increased by 10.1 h, the fuel consumption is reduced by 92.24 tons, saving 6.94%. Compared with the traditional genetic algorithm, the voyage distance and time are reduced by 216.93 nautical miles and 7.5 h, and the fuel consumption is reduced by 58.82 tons, which is almost 4.54%. Through experimental verification, the proposed model can obtain punctual routes, avoid areas with bad sea conditions, reduce fuel consumption, and is of great significance for improving the safety and economy of ship routes.

show abstract

An Optimized Path Planning Method for Container Ships in Bohai Bay Based on Improved Deep Q-Learning

Cited by 4 publications

References 37 publications

Ship collision avoidance decision-making research in coastal waters considering uncertainty of target ships

Ship collision avoidance decision-making research in coastal waters considering uncertainty of target ships

Path Planning for Outdoor Mobile Robots Based on IDDQN

Ship Global Traveling Path Optimization via a Novel Non-Dominated Sorting Genetic Algorithm

Contact Info

Product

Resources

About