A review on learning to solve combinatorial optimisation problems in manufacturing

Abstract

An efficient manufacturing system is key to maintaining a healthy economy today. With the rapid development of science and technology and the progress of human society, the modern manufacturing system is becoming increasingly complex, posing new challenges to both academia and industry. Ever since the beginning of industrialisation, leaps in manufacturing technology have always accompanied technological breakthroughs from other fields, for example, mechanics, physics, and computational science. Recently, machine learning (ML) technology, one of the crucial subjects of artificial intelligence, has made remarkable progress in many areas. This study thoroughly reviews how ML, specifically deep (reinforcement) learning, motivates new ideas for addressing challenging problems in manufacturing systems. We collect the literature targeting three aspects: scheduling, packing, and routing, which correspond to three pivotal cooperative production links of today's manufacturing system, that is, production, packing, and logistics respectively. For each aspect, we first present and discuss the state-of-the-art research. Then we summarise and analyse the development trends and point out future research opportunities and challenges. K E Y W O R D Sbin packing, combinatorial optimisation, deep reinforcement learning, job shop scheduling, manufacturing systems, vehicle routing | INTRODUCTIONCombinatorial optimisation problems (COPs), as one important branch of mathematical optimisation, have practical applications in many fields, such as communication, transportation, manufacturing and aroused broad research in industrial engineering, computer science, and operations research. Due to the NP (non-deterministic polynomial-time) hardness, finding their optimal solutions is challenging. In specific, the discrete solution space in COPs renders the optimisation less efficient, without the guidance of gradient as in continuous optimisation. Meanwhile, the complexity of searching the (near-)optimal solution(s) among feasible solutions could exponentially increase as the problem scale grows. Classic methods, including exact algorithms and (meta-)heuristics, generally depend on massive expertise and tuning work to solve specific problems. They are Cong Zhang, Yaoxin Wu, and Yining Ma are equal contribution.

Research on vehicle path planning of automated guided vehicle with simultaneous pickup and delivery with mixed time windows

“…-JIANG ET AL. 3. Experimental results show the AGA-VNS algorithm is not only robustly applicable to AGV distributing tasks in intelligent workshops but also surpasses current heuristic algorithms in solution quality.…”

“…Constraint (2) ensures that the number of active AGVs is always less than or equal to the total number of AGVs. Constraint (3) ensures that each customer must be visited exactly once by one AGV and that AGV also leaves after this visit. Constraint ( 4) is used to eliminate subloops.…”

“…An increasing number of scholars are beginning to focus on the study of various optimisation problems in the manufacturing industry [3]. Since Dantzig and Ramser [4] first introduced the VRP, it has continually garnered substantial academic interest.…”

Learning Encodings for Constructive Neural Combinatorial Optimization Needs to Regret

“…Neural combinatorial optimization (NCO) has recently demonstrated its superiority in dealing with various COPs (Kool, van Hoof, and Welling 2019;Drakulic et al 2023;Luo et al 2023;Huang et al 2023;Li, Zhang, and Wang 2021;Zhang et al 2023). Especially, deep reinforcement learning (DRL) based methods demonstrate excellent performance without utilizing the optimal solution as groundtruth labels for model training (Bello et al 2017;Chen and Tian 2019;Wu et al 2022;Ma et al 2021;Wang et al 2023).…”

Residual Scheduling: A New Reinforcement Learning Approach to Solving Job Shop Scheduling Problem

“…Recently, deep reinforcement learning (DRL) has made several significant successes for some applications, such as AlphaGo [15], AlphaStar [16], AlphaTensor [17], and thus it also attracted much attention in the CO problems, including chip design [18] and scheduling problems [19]. In the past, several researchers used DRL methods as construction heuristics, and their methods did improve scheduling performance, illustrated as follows.…”