Scheduling Storage Process of Shuttle-Based Storage and Retrieval Systems Based on Reinforcement Learning

Abstract

The Shuttle-Based Storage and Retrieval System (SBS/RS) has been widely studied because it is currently the most efficient automated warehousing system. Most of the related existing studies are focused on the prediction and improvement of the efficiency of such a system at the design stage. Hence, the control of existing SBS/RSs has been rarely investigated. In existing SBS/RSs, some empirical rules, such as storing loads column by column, are used to control or schedule the storage process. The question is whether or not the control of the storage process in an existing system can be improved further by using a different approach. The storage process is controlled to minimize the makespan of storing a series of loads into racks. Empirical storage rules are easy to control, but they do not reach the minimum makespan. In this study, the performance of a control system that uses reinforcement learning to schedule the storage process of an SBS/RS with fixed configurations is evaluated. Specifically, a reinforcement learning algorithm called the actor-critic algorithm is used. This algorithm is made up of two neural networks and is effective in making decisions and updating itself. It can also reduce the makespan relative to the existing empirical rules used to improve system performance. Experiment results show that in an SBS/RS comprising six columns and six tiers and featuring a storage capacity of 72 loads, the actor-critic algorithm can reduce the makespan by 6.67% relative to the column-by-column storage rule. The proposed algorithm also reduces the makespan by more than 30% when the number of loads being stored is in the range of 7-45, which is equal to 9.7%-62.5% of the systems' storage capacity.

A survey on federated learning: challenges and applications

“…Considering from this perspective, Liu et al [76] studied an orthogonal method to identify irrelevant updates made by clients by detecting whether local model updates are consistent with the trend of global model updates, so as to avoid transmitting those irrelevant parameters to the server and reduce the occupation of network resources. Deng et al [77] proposed an automated-quality-awareness based client selection framework, which taken into account various factors affecting learning performance, adopted reinforcement learning method [78,79]. And it automatically evaluated the model quality of clients, and automatically selects highquality clients for the federated task within a limited budget, which significantly improves learning efficiency.…”

Dynamic Scheduling Algorithm Based on Evolutionary Reinforcement Learning for Sudden Contaminant Events Under Uncertain Environment

“…By combining the learning and expression mechanism of deep learning, DRL fully exploits both the decision-making and perceptual abilities of complex control problems. It has been widely used in many fields [12,13] . Evolutionary reinforcement learning (ERL) is a hybrid algorithm which combines the advantages of evolutionary computation (EC) and reinforcement learning (RL), including two stages of population optimization and agent policy update.…”

Modeling and performance analysis of shuttle-based compact storage systems under parallel processing policy

“…Lei et al [ 17 ] investigate the optimal storage location assignment by using a optimization model. Besides, Luo et al [ 18 ] and Dong et al [ 19 ] investigate the optimal scheduling rule for storage and retrieval processes, respectively, to minimize the makespan of storing or retrieving a series of loads. And Liu et al [ 20 ] develops an energy consumption model for the SBS/RS and estimate the maximum energy consumption under different throughput requirement.…”