Anti-conflict AGV path planning in automated container terminals based on multi-agent reinforcement learning

Hu, Hongtao; Yang, Xurui; Xiao, Shichang; Wang, Feiyang

doi:10.1080/00207543.2021.1998695

Cited by 62 publications

(33 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL has several application areas in manufacturing, such as dynamic scheduling of tasks in cloud manufacturing [52], maintenance scheduling [53], path planning of automated guided vehicles [54], or human-robot interaction [55]. Furthermore, RL has been successfully used for multi objective optimization [56].…”

Section: Reinforcement Learningmentioning

confidence: 99%

Multi objective factory layout planning using simulation-based reinforcement learning

Klar¹,

Schworm²,

Wu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Factory layout planning aims at finding an optimized layout configuration under consideration of varying influences such as the material flow characteristics. Manual layout planning can be characterized as a complex decision-making process due to a large number of possible placement options. Automated planning approaches aim at reducing the manual planning effort by generating optimized layout variants in the early stages of layout planning. Recent developments have introduced Reinforcement Learning (RL) based planning approaches that allow to optimize a layout under consideration of a single optimization criterion. However, within layout planning, multiple partially conflicting planning objectives have to be considered. Such multiple objectives are not considered by existing RL-based approaches. This paper addresses this research gap by presenting a novel RL-based layout planning approach that allows consideration of multiple objectives for optimization. Furthermore, existing RL-based planning approaches only consider analytically formulated objectives such as the transportation distance. Consequently, dynamic influences in the material flow are neglected which can result in higher operational costs of the future factory. To address this issue, a discrete event simulation module is developed that allows simulating manufacturing and material flow processes simultaneously for any layout configuration generated by the RL approach. Consequently, the presented approach considers material flow simulation results for multi-objective optimization. In order to investigate the capabilities of RL-based factory layout planning, different RL architectures are compared based on a simplified application scenario. In terms of optimization objectives, the throughput time, media supply, and clarity of the material flow are considered. The best performing architecture is then applied to an industrial planning scenario with 43 functional units to illustrate the approach. Furthermore, the performance of the RL approach is compared to the manually planned layout and to the results generated by a combined version of the genetic algorithm and tabu search. The results indicate that the RL approach is capable of improving the manually planned layout significantly. Furthermore, it reaches comparable results for the throughput time and better results for the clarity of the material flow compared to the combined version of a genetic algorithm and tabu search.

show abstract

Section: Reinforcement Learningmentioning

confidence: 99%

Multi objective factory layout planning using simulation-based reinforcement learning

Klar¹,

Schworm²,

Wu³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Compared with manually operated vehicles, automated vehicles can be better managed and controlled at both planning and operational levels. Using AGV as an example, many studies have been conducted on path planning ( Zhong et al, 2020 ; Hu et al, 2021 ), dispatching ( Hu et al, 2019 ; Wang and Zeng, 2022 ), scheduling ( Chen et al, 2020 ; Hu et al, 2022 ), and coordination between AGV and other terminal equipment ( Yang et al, 2018a ). Among all the research problems, deadlock- and collision-free scheduling and path planning for large scale vehicle fleet are the most challenging part in AGV operations, as the operators prefer to guarantee safety with less throughput rather than go to the ground to resolve physical issues.…”

Section: Trends In Emerging Technology and Management Researchmentioning

confidence: 99%

Emerging technology and management research in the container terminals: Trends and the COVID-19 pandemic impacts

Zhou¹,

Zhu²,

Bell³

et al. 2022

Ocean & Coastal Management

View full text Add to dashboard Cite

“…Fanti et al [10] proposed a decentralized control-based coordination algorithm that first assigns tasks to AGVs through a consensus-based approach, and then schedules AGVs to move in a proposed path network based on a regional control strategy, avoiding collisions and deadlock-free behavior. To find the shortest path and avoid the conflicts of AGVs, Hu et al [11] studied a multi-agent reinforcement learning method which scored and learned the selection of paths.…”

Section: Related Workmentioning

confidence: 99%

“…Constraint (10) shows that the decelerating and accelerating time of AGV when meeting conflicts. Constraints (11) and ( 12) are the integer restrictions of the decision variables.…”

Section: Model Formulationmentioning

confidence: 99%

Machine Learning and Digital Twin-Based Path Planning for AGVs at Automated Container Terminals

Gao

Chen

Chang

et al. 2022

Advances in Transdisciplinary Engineering

View full text Add to dashboard Cite

The complex and dynamic environment in an automated container terminal (ACT) increases the difficulty of path planning, especially for automated guided vehicles (AGVs). Digital twin is an essential means of characterizing complex production systems as the physical objects can be synchronized in the virtual space. Machine learning is also a popular way to solve path planning problems. This study combines digital twin and machine learning to tackle AGV path planning problems in the time-changing operation environment. A digital twin-based AGV scheduling approach is developed to obtain the real-time data from the physical ACT. Based on the information of the dynamic factors obtained, a mathematical model is formulated to minimize the transportation time of AGVs. Subsequently, the path planning schemes solved by machine learning are used asinput to the virtual ACT for validation and optimization. The optimized solution is further compared to a common path plan algorithm without applying digital twin. The experimental results show the superiority of the proposed method, which can provide better decision support for ACT operation.

show abstract

Anti-conflict AGV path planning in automated container terminals based on multi-agent reinforcement learning

Cited by 62 publications

References 30 publications

Multi objective factory layout planning using simulation-based reinforcement learning

Multi objective factory layout planning using simulation-based reinforcement learning

Emerging technology and management research in the container terminals: Trends and the COVID-19 pandemic impacts

Machine Learning and Digital Twin-Based Path Planning for AGVs at Automated Container Terminals

Contact Info

Product

Resources

About