Automatic collision avoidance of multiple ships based on deep Q-learning

Shen, Haiqing; Hashimoto, Hirotada; Matsuda, Akihiko; Taniguchi, Yousuke; Terada, Daisuke; Guo, Chen

doi:10.1016/j.apor.2019.02.020

Cited by 148 publications

(51 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These rules have been implemented as components of the cost function or as penalty functions. Some papers emphasise that, according to good seamanship practice, course change is preferred over speed change in collision avoidance scenarios [38,49].…”

Section: Properties Of Algorithmsmentioning

confidence: 99%

Path planning and collision avoidance for autonomous surface vehicles II: a comparative study of algorithms

et al. 2021

View full text Add to dashboard Cite

Artificial intelligence is an enabling technology for autonomous surface vehicles, with methods such as evolutionary algorithms, artificial potential fields, fast marching methods, and many others becoming increasingly popular for solving problems such as path planning and collision avoidance. However, there currently is no unified way to evaluate the performance of different algorithms, for example with regard to safety or risk. This paper is a step in that direction and offers a comparative study of current state-of-the art path planning and collision avoidance algorithms for autonomous surface vehicles. Across 45 selected papers, we compare important performance properties of the proposed algorithms related to the vessel and the environment it is operating in. We also analyse how safety is incorporated, and what components constitute the objective function in these algorithms. Finally, we focus on comparing advantages and limitations of the 45 analysed papers. A key finding is the need for a unified platform for evaluating and comparing the performance of algorithms under a large set of possible real-world scenarios.

show abstract

Section: Properties Of Algorithmsmentioning

confidence: 99%

Path planning and collision avoidance for autonomous surface vehicles II: a comparative study of algorithms

et al. 2021

View full text Add to dashboard Cite

show abstract

“…For artificial intelligence algorithms, two neuroevolutionary methods were used to build a collision avoidance system of USVs in [15]. Deep reinforcement learning algorithms [16,17] have also recently been proposed for USV collision avoidance. These algorithms can achieve good collision avoidance effects in some situations.…”

Section: Introductionmentioning

confidence: 99%

Local Collision Avoidance Algorithm for a Unmanned Surface Vehicle Based on Steering Maneuver Considering COLREGs

Wang¹,

Zhang²,

Jin³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

In order to sail safely in a marine environment with neighboring obstacles, a local collision avoidance algorithm based on a steering maneuver considering International Regulations for Preventing Collisions at Sea (COLREGs) is proposed for unmanned surface vehicles (USVs). In this paper, the algorithm consists of three parts: collision risk assessment, steering occasion determination, and navigation waypoint update. The collision risk assessment is used to judge a collision risk based on an analytical feasible angle interval by the closest point of approach method. The steering occasion determination provides a safe distance interval from an obstacle by analyzing a marginal steering angular velocity. The navigation waypoint update generates a temporary waypoint to navigate the USV and considers the overtaking, head-on, and crossing rules of COLREGs. The temporary waypoint is calculated based on a real-time optimal orientation angle, which is determined by an optimization objective function for angle deviation. The three parts for local collision avoidance are determined by a finite state machine. In simulations, a USV with actual dynamic constraints runs successfully in a static and dynamic multiobstacle environment, and the results indicate the feasibility and validity of the proposed algorithm.

show abstract

“…The simulation results showed that the unmanned ships could successfully avoid obstacles and reached the destination in a complex environment. Shen, H.Q et al [30] offered a method that was based on the Dueling DQN algorithm for automatic collision avoidance of multiple ships, and combined ship manoeuvrability, crew's experience, and COLREGS to verify the path planning and collision avoidance capability of unmanned ships. Zhang, R.B et al [31], based on the Sarsa on-policy algorithm, proposed a behavior-based USV local path planning and obstacle avoidance method, and tested in real marine environment.…”

Section: Introductionmentioning

confidence: 99%

An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning

Guo

Zhang

Zheng

et al. 2020

Sensors

130

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) has excellent performance in continuous control problems and it is widely used in path planning and other fields. An autonomous path planning model based on DRL is proposed to realize the intelligent path planning of unmanned ships in the unknown environment. The model utilizes the deep deterministic policy gradient (DDPG) algorithm, through the continuous interaction with the environment and the use of historical experience data; the agent learns the optimal action strategy in a simulation environment. The navigation rules and the ship’s encounter situation are transformed into a navigation restricted area, so as to achieve the purpose of planned path safety in order to ensure the validity and accuracy of the model. Ship data provided by ship automatic identification system (AIS) are used to train this path planning model. Subsequently, the improved DRL is obtained by combining DDPG with the artificial potential field. Finally, the path planning model is integrated into the electronic chart platform for experiments. Through the establishment of comparative experiments, the results show that the improved model can achieve autonomous path planning, and it has good convergence speed and stability.

show abstract

Automatic collision avoidance of multiple ships based on deep Q-learning

Cited by 148 publications

References 14 publications

Path planning and collision avoidance for autonomous surface vehicles II: a comparative study of algorithms

Path planning and collision avoidance for autonomous surface vehicles II: a comparative study of algorithms

Local Collision Avoidance Algorithm for a Unmanned Surface Vehicle Based on Steering Maneuver Considering COLREGs

An Autonomous Path Planning Model for Unmanned Ships Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About