Improved reinforcement learning for collision-free local path planning of dynamic obstacle

Han, Qilong

doi:10.1016/j.oceaneng.2023.115040

Cited by 10 publications

(4 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AI-based methods, such as reinforcement learning (RL) [112][113][114] and self-organizing maps (SOMs) [115][116][117], have demonstrated remarkable strengths that elevate their effectiveness. On the one hand, RL algorithms can learn optimal paths by iteratively interacting with an environment, receiving feedback in the form of rewards, see Figure 10.…”

Section: Ai-based Methodsmentioning

confidence: 99%

Unlocking the Ocean 6G: A Review of Path-Planning Techniques for Maritime Data Harvesting Assisted by Autonomous Marine Vehicles

Zhao,

Bai

2024

JMSE

View full text Add to dashboard Cite

Seamless integration of both terrestrial and non-terrestrial networks is crucial to providing full-dimensional wireless and ubiquitous coverage, particularly catering to those engaged in marine activities. Compared to terrestrial networks, wireless communications in the marine domain are still not satisfactory for ubiquitous connectivity. Featuring agile maneuverability and strong adaptive capability, autonomous marine vehicles (AMVs) play a pivotal role in enhancing communication coverage by relaying or collecting data. However, path planning for maritime data harvesting is one of the most critical issues to enhance transmission efficiency while ensuring safe sailing for AMVs; yet it has rarely been discussed under this context. This paper provides a comprehensive and holistic overview of path-planning techniques custom-tailored for the purpose of maritime data collection. Specifically, we commence with a general portrayal of fundamental models, including system architectures, problem formulations, objective functions, and associated constraints. Subsequently, we summarize the various algorithms, methodologies, platforms, tools, coding environments, and their practical implementations for addressing these models. Furthermore, we delve into the burgeoning applications of path planning in the realm of maritime data harvesting and illuminate potential avenues for upcoming research endeavors. We believe that future research may focus on developing techniques to adapt more intricate and uncertain scenarios, such as sensor failures, inaccurate state estimations, complete modeling of communication channels, ocean dynamics, and application of heterogeneous systems.

show abstract

Section: Ai-based Methodsmentioning

confidence: 99%

Unlocking the Ocean 6G: A Review of Path-Planning Techniques for Maritime Data Harvesting Assisted by Autonomous Marine Vehicles

Zhao,

Bai

2024

JMSE

View full text Add to dashboard Cite

show abstract

“…Furthermore, the terms, i.e., control system, multi-agent systems, nonlinear-systems and tracking control indicated researchers' utilization of real-time or historical traffic data for developing novel systems grounded in AI. This shift could be linked to the progressive integration of AI in the shipping industry, as evidenced by relevant studies [60][61][62][63][64][65][66][67][68][69][70]. Additionally, the inclusion of environments, identification, remote sensing and object detection in the ranking suggested a growing inclination towards environmental perception in maritime decision-making process.…”

Section: Abstract Analysismentioning

confidence: 99%

A Bibliometric Analysis and Overall Review of the New Technology and Development of Unmanned Surface Vessels

Yang,

Xue,

2024

JMSE

View full text Add to dashboard Cite

With the significant role that Unmanned Surface Vessels (USVs) could play in industry, the military and the transformation of ocean engineering, a growing research interest in USVs is attracted to their innovation, new technology and automation. Yet, there has been no comprehensive review grounded in bibliometric analysis, which concentrates on the most recent technological advancements and developments in USVs. To provide deeper insight into the relevant research trends, this study employs a bibliometric analysis to examine the basic features of the literature from 2000 to 2023, and identifies the key research hotspots and modeling techniques by reviewing their current statuses and the recent efforts made in these areas. Based on the analysis of the temporal and spatial trends, disciplines and journals’ distribution, institutions, authors and citations, the publications relating to the new technology of USVs are assessed based on their keywords and the term analysis in the literature; six future research directions are proposed, including enhanced intelligence and autonomy, highly integrated sensor systems and multi-modal task execution, extended endurance and resilience, satellite communication and interconnectivity, eco-friendly and sustainable practices and safety and defense. The scientific literature is reviewed in a systematic way using a comparative analysis of existing tools, and the results greatly contribute to understanding the overall situation of new technology in USVs. This paper is enlightening to students, international scholars and institutions, as it can facilitate partnerships between industry and academia to allow for concerted efforts to be made in the domain of USVs.

show abstract

“…In the domain of local path planning for aviation, a diverse array of algorithms has been implemented, each characterized by its unique methodology and specific applications. These encompass the artificial potential field method, the guidance method, machine learning-based obstacle avoidance algorithms, the VO method, and the dynamic window approach [6,7]. The artificial potential field method, functioning through the calculation of gravitational and repulsive forces to navigate around obstacles, faces certain limitations.…”

Section: Introductionmentioning

confidence: 99%

Swarm Division-Based Aircraft Velocity Obstacle Optimization Considering Low-Carbon Emissions

Zhong,

Yu,

Zhang

et al. 2024

Sustainability

View full text Add to dashboard Cite

In the pursuit of sustainable aviation, this paper presents an innovative approach that adopts a swarm division strategy to enhance and refine the velocity obstacle (VO) method, guided by a low-carbon principle. A dynamic elliptical protection zone model forms the core of this innovative approach. Specifically, this dynamic elliptical protection zone is created based on the difference in aircraft velocity, and a swarm division strategy is introduced in this process. Initially, aircraft that share the same route and type, and have similar velocities and distances, are grouped into swarms. Then, the characteristics of the swarms, such as mass points, velocities, and protection zones, are recorded. Second, the collision cone (CC) between swarms is established, and planar geometrical analysis is used to determine the optimal relief velocity and heading of aircraft on the low-carbon objective while ensuring a safe interval between aircraft in the swarm during the relief period. Additionally, a swarm control algorithm is utilized to adjust the velocity of the aircraft by a small margin. Finally, simulation experiments are conducted using Python, revealing that the swarm relief efficiency of the enhanced VO method sees a notable increase of over 33%. Concurrently, the need for adjustments decreases by an average of 32.78%, while fuel savings reach as high as 70.18%. The strategy is real-time and operational, significantly reduces the air traffic controller (ATC) workload, improves flight efficiency and safety, and contributes positively to the reduction in carbon emissions, which is beneficial for the environment.

show abstract

Improved reinforcement learning for collision-free local path planning of dynamic obstacle

Cited by 10 publications

References 38 publications

Unlocking the Ocean 6G: A Review of Path-Planning Techniques for Maritime Data Harvesting Assisted by Autonomous Marine Vehicles

Unlocking the Ocean 6G: A Review of Path-Planning Techniques for Maritime Data Harvesting Assisted by Autonomous Marine Vehicles

A Bibliometric Analysis and Overall Review of the New Technology and Development of Unmanned Surface Vessels

Swarm Division-Based Aircraft Velocity Obstacle Optimization Considering Low-Carbon Emissions

Contact Info

Product

Resources

About