Learning to Solve Multiple-TSP With Time Window and Rejections via Deep Reinforcement Learning

Zhang, Rongkai; Zhang, Cong; Cao, Zhiguang; Song, Wen; Tan, Puay Siew; Zhang, Jie; Wen, Bihan; Dauwels, Justin

doi:10.1109/tits.2022.3207011

Cited by 30 publications

(47 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Zhang et al. [168], a more practical TSPTW variant called multiple‐vehicle TSP with time window and rejections was studied, where customers can be rejected due to TW constraint.…”

Section: Vehicle Routing Problemsmentioning

confidence: 99%

“…By proposing a novel Multi-Agent Attention Model, Zhang et al [150] tackled the VRP with Soft Time Windows (VRPSTW) and outperformed OR-Tools with shorter computation time. In Zhang et al [168], a more practical TSPTW variant called multiple-vehicle TSP with time window and rejections was studied, where customers can be rejected due to TW constraint.…”

Section: Classical Routing Problemsmentioning

confidence: 99%

See 1 more Smart Citation

A review on learning to solve combinatorial optimisation problems in manufacturing

Zhang

et al. 2023

IET Collab Intel Manufact

Self Cite

View full text Add to dashboard Cite

An efficient manufacturing system is key to maintaining a healthy economy today. With the rapid development of science and technology and the progress of human society, the modern manufacturing system is becoming increasingly complex, posing new challenges to both academia and industry. Ever since the beginning of industrialisation, leaps in manufacturing technology have always accompanied technological breakthroughs from other fields, for example, mechanics, physics, and computational science. Recently, machine learning (ML) technology, one of the crucial subjects of artificial intelligence, has made remarkable progress in many areas. This study thoroughly reviews how ML, specifically deep (reinforcement) learning, motivates new ideas for addressing challenging problems in manufacturing systems. We collect the literature targeting three aspects: scheduling, packing, and routing, which correspond to three pivotal cooperative production links of today's manufacturing system, that is, production, packing, and logistics respectively. For each aspect, we first present and discuss the state-of-the-art research. Then we summarise and analyse the development trends and point out future research opportunities and challenges. K E Y W O R D Sbin packing, combinatorial optimisation, deep reinforcement learning, job shop scheduling, manufacturing systems, vehicle routing | INTRODUCTIONCombinatorial optimisation problems (COPs), as one important branch of mathematical optimisation, have practical applications in many fields, such as communication, transportation, manufacturing and aroused broad research in industrial engineering, computer science, and operations research. Due to the NP (non-deterministic polynomial-time) hardness, finding their optimal solutions is challenging. In specific, the discrete solution space in COPs renders the optimisation less efficient, without the guidance of gradient as in continuous optimisation. Meanwhile, the complexity of searching the (near-)optimal solution(s) among feasible solutions could exponentially increase as the problem scale grows. Classic methods, including exact algorithms and (meta-)heuristics, generally depend on massive expertise and tuning work to solve specific problems. They are Cong Zhang, Yaoxin Wu, and Yining Ma are equal contribution.

show abstract

“…In Zhang et al. [168], a more practical TSPTW variant called multiple‐vehicle TSP with time window and rejections was studied, where customers can be rejected due to TW constraint.…”

Section: Vehicle Routing Problemsmentioning

confidence: 99%

Section: Classical Routing Problemsmentioning

confidence: 99%

A review on learning to solve combinatorial optimisation problems in manufacturing

Zhang

et al. 2023

IET Collab Intel Manufact

Self Cite

View full text Add to dashboard Cite

show abstract

“…The algorithm was tested for generalization ability by training the network with random events in the existing environment to quickly adapt to new problems. However, the agent was not tested with a completely new dataset, an issue which was overcome by Zhang et al [2020a]. The authors developed a graph neural network which enabled them to solve size-agnostic problems.…”

Section: Related Workmentioning

confidence: 99%

“…The action space was designed for n jobs along with an additional job called No-Op (No Operation). The agent was tested with different benchmark instances where the agent performed around 18% better than Zhang et al [2020a] and 10% better than Han and Yang [2020]. Even though they provide a near-optimum solution, the approach falls short of the generalization objective.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Reinforcement Learning Approach for Scheduling Problems With Improved Generalization Through Order Swapping

Vivekanandan¹,

Wirth²,

Karlbauer³

et al. 2023

Preprint

View full text Add to dashboard Cite

The scheduling of production resources (such as associating jobs to machines) plays a vital role for the manufacturing industry not only for saving energy but also for increasing the overall efficiency. Among the different job scheduling problems, the Job Shop Scheduling Problem (JSSP) is addressed in this work. JSSP falls into the category of NP-hard Combinatorial Optimization Problem (COP), in which solving the problem through exhaustive search becomes unfeasible. Simple heuristics such as First In, First Out (FIFO), Largest Processing Time First (LPT) and metaheuristics such as Taboo search are often adopted to solve the problem by truncating the search space. The viability of the methods becomes inefficient for large problem sizes as it is either far from the optimum or time consuming. In recent years, the research towards using Deep Reinforcement Learning (DRL) to solve COPs has gained interest and has shown promising results in terms of solution quality and computational efficiency. In this work, we provide an novel approach to solve the JSSP examining the objectives generalization and solution effectiveness using DRL. In particular, we employ the Proximal Policy Optimization (PPO) algorithm that adopts the policy-gradient paradigm that is found to perform well in the constrained dispatching of jobs. We incorporated an Order Swapping Mechanism (OSM) in the environment to achieve better generalized learning of the problem. The performance of the presented approach is analyzed in depth by using a set of available benchmark instances and comparing our results with the work of other groups.

show abstract

Deep Reinforcement Learning for Multiobjective Scheduling in Industry 5.0 Reconfigurable Manufacturing Systems

Bezoui,

Kermali,

Bounceur

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In modern-day manufacturing, it is imperative to react promptly to altering market requirements. Reconfigurable Manufacturing Systems (RMS) are a significant leap forward in achieving this criteria as they offer a flexible and affordable structure to comply with evolving production necessities. The ever-changing nature of RMS demands a sturdy induction of learning algorithms to persistently improve system configurations and scheduling. This study suggests that using Reinforcement Learning (RL), specifically, the Double Deep Q-Network (DDQN) algorithm, is a feasible way to navigate the intricate, multi-objective optimization landscape of RMS. Key points to consider regarding this study include cutting down tardiness costs, ensuring sustainability by reducing wasted liquid and gas emissions during production, optimizing makespan, and improving ergonomics by reducing operator intervention during system reconfiguration. Our proposal consists of two layers. Initially, we suggest a hierarchical and modular architecture for RMS which includes a multi-agent environment at the reconfigurable machine tool level, which improves agent interaction for optimal global results. Secondly, we incorporate DDQN to navigate the multi-objective space in a clever manner, resulting in more efficient and ergonomic reconfiguration and scheduling. The findings indicate that employing RL can help solve intricate optimization issues that come with contemporary manufacturing paradigms, clearing the path for Industry 5.0.

show abstract

Learning to Solve Multiple-TSP With Time Window and Rejections via Deep Reinforcement Learning

Cited by 30 publications

References 22 publications

A review on learning to solve combinatorial optimisation problems in manufacturing

A review on learning to solve combinatorial optimisation problems in manufacturing

A Reinforcement Learning Approach for Scheduling Problems With Improved Generalization Through Order Swapping

Deep Reinforcement Learning for Multiobjective Scheduling in Industry 5.0 Reconfigurable Manufacturing Systems

Contact Info

Product

Resources

About