An Intelligent Marshalling Plan Using a New Reinforcement Learning System for Container Yard Terminals

Hirashima, Yoichi

doi:10.5772/6281

Cited by 3 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite the recent achievements of deep Reinforcement Learning (deep RL) in complex gaming environments like Go [3] and for increasing energy efficiency of cooling systems [4], the use of RL for scheduling problems has been limited [5,6]. To the best of our knowledge ours is the first attempt to solve container loading problem using deep RL, while it has been used for other problems of port operations such as container allocation in the yard [6] etc. Similar to RL, evolutionary strategies have also been used to solve similar problems [7].…”

Section: Introductionmentioning

confidence: 99%

Evolutionary RL for Container Loading

Saikia,

Verma,

Agarwal

et al. 2018

Preprint

View full text Add to dashboard Cite

Loading the containers on the ship from a yard, is an important part of port operations. Finding the optimal sequence for the loading of containers, is known to be computationally hard and is an example of combinatorial optimization, which leads to the application of simple heuristics in practice. In this paper, we propose an approach which uses a mix of Evolutionary Strategies and Reinforcement Learning (RL) techniques to find an approximation of the optimal solution. The RL based agent uses the Policy Gradient method, an evolutionary reward strategy and a Pool of good (not-optimal) solutions to find the approximation. We find that the RL agent learns near-optimal solutions that outperforms the heuristic solutions. We also observe that the RL agent assisted with a pool generalizes better for unseen problems than an RL agent without a pool. We present our results on synthetic data as well as on subsets of real-world problems taken from container terminal. The results validate that our approach does comparatively better than the heuristics solutions available, and adapts to unseen problems better.

show abstract

Section: Introductionmentioning

confidence: 99%

Evolutionary RL for Container Loading

Saikia,

Verma,

Agarwal

et al. 2018

Preprint

View full text Add to dashboard Cite

show abstract

Locating algorithm of steel stock area with asynchronous advantage actor-critic reinforcement learning

Cho,

Kim,

Yoon

et al. 2023

Journal of Computational Design and Engineering

View full text Add to dashboard Cite

In the steel stockyard of the shipyard, the sorting work to relocate the steel plates already stacked to retrieve the target steel plate on the fabrication schedule is labor-consuming work requiring the operation of overhead cranes. To reduce the sorting work, there is a need for a method of stacking the steel plates in order of fabrication schedules when the steel plates arrive at the shipyard from the steel-making companies. However, the conventional optimization algorithm and heuristics have limitations in determining the optimal stacking location of steel plates because the real-world stacking problems in shipyards have vast solution space in addition to the uncertainty in the arrival order of steel plates. In this study, reinforcement learning is applied to the development of a real-time stacking algorithm for steel plates considering the fabrication schedule. Markov decision process suitable for the stacking problem is defined, and the optimal stacking policy is learned using an asynchronous advantage actor-critic algorithm. The learned policy is tested on several problems by varying the number of steel plates. The test results indicate that the proposed method is effective for minimizing the use of cranes compared with other metaheuristics and heuristics for stacking problems.

show abstract

Machine Learning (Reinforcement Learning)-Based Steel Stock Yard Planning Algorithm

Woo

Cho

et al. 2020

2020 Winter Simulation Conference (WSC)

View full text Add to dashboard Cite

An Intelligent Marshalling Plan Using a New Reinforcement Learning System for Container Yard Terminals

Cited by 3 publications

References 8 publications

Evolutionary RL for Container Loading

Evolutionary RL for Container Loading

Locating algorithm of steel stock area with asynchronous advantage actor-critic reinforcement learning

Machine Learning (Reinforcement Learning)-Based Steel Stock Yard Planning Algorithm

Contact Info

Product

Resources

About