A Q-Learning for Group-Based Plan of Container Transfer Scheduling

Abstract

In container yard terminals, containers brought by trucks arrive in the random order. Since each container has its own destination and it cannot be rearranged after shipping, containers have to be loaded into a ship in a certain order. Therefore, containers have to be rearranged from the initial arrangement into the desired arrangement before shipping. In the problem, the number of container-arrangements increases by the exponential rate with increase of total count of containers. Therefore, conventional methods have great difficulties to determine desirable movements of containers in order to reduce the run time for shipping. In this paper, a Q-Learning algorithm based on the number of container-movements for the material handling in the container yard terminal is proposed. In the proposed method, each container has several desired positions, so that the learning performance can be improved. In order to show effectiveness of the proposed method, simulations for several examples are conducted.

Online rules for container stacking

“…Nevertheless, finding a theoretical optimum can be very useful as a benchmark (although the metaheuristic approaches used to make this approach computationally feasible are not guaranteed to find the global optimum). Other methods that have been used for this include Q-Learning (Hirashima et al 2006) and critical-shaking neighborhood search (Lim and Xu 2006). Han et al (2008) use integer programming with Tabu search to generate an entire yard template, which should minimize reshuffling moves.…”

Rolling Cargo Management Using a Deep Reinforcement Learning Approach

“…The yard crane operators were modeled as agents using the Q-learning technique for automatic trailer selection process. In the same context, Reference [3] used the Q-learning algorithm to determine the movement sequence of the containers so that they were loaded onto a ship in the desired order. Their ultimate goal was to reduce the run time for shipping.…”

“…Orive et al [2] gave a thorough analysis of the conversion to digital, intelligent, and green ports and proposed a new methodology, called the business observation tool, that allows successfully undertaking the automation of terminals considering the specific constraints of the port. Furthermore, various logistics operations have been studied using new techniques such as cargo management, traffic control, detection, recognition and tracking of traffic-related objects, and optimization of different resources such as energy, time, and materials [3,4]. The state-of-the-art of smart ports and intelligent maritime transportation has been described highlighting trends, challenges, new technologies, and their implementation status [5][6][7].…”

Inland waterway terminal yard configuration contributing to sustainability: Modeling yard operations

“…However, their conclusions are useful for the determination of the average number of reshuffles relative to the productive moves at the terminal. Hirashima et al (2006) focus on the pre-marshaling process for export containers. In the arrangement problem, the number of container-arrangements increases exponentially with increasing container volume.…”