You Only Train Once: A highly generalizable reinforcement learning method for dynamic job shop scheduling problem

Abstract

<p>Research in artificial intelligence demonstrates the applicability and flexibility of the reinforcement learning (RL) technique for the dynamic job shop scheduling problem (DJSP). However, the RL-based method will always overfit to the training environment and cannot generalize well to novel unseen situations at deployment time, which is unacceptable in real-world production. For this reason, this paper proposes a highly generalizable reinforcement learning framework named Train Once For All (TOFA) for the dynamic job shop scheduling problem. The trivial and non-trivial states are distinguished when the DJSP is formulated as a semi-Markov decision process, defining the size-agnostic state, action, and reward function. A novel graph representation learning method based on attention mechanism and spatial pyramid pooling is implemented to compress the disjunctive graphs of differentsize DJSP into fixed-length feature vectors. Combining the proposed dynamic frame skipping and an improved prioritized experience replay method that considers the sample quality difference at different training phases. TOFA shows superb generalization capability, outperforms practically favored dispatching rules and even instance-by-instance training RL-based schedulers on various benchmark DJSP. Additionally, we proved that TOFA acquires a transferable scheduling policy that can be used to schedule a whole new DJSP without additional training.</p>

A review on learning to solve combinatorial optimisation problems in manufacturing

“…Some methods start considering various dynamic events to make the scheduler more robust to these disturbances. Zeng et al [94] consider machine breakdown and different order requirements in JSSP, where a machine can break or the configuration of jobs may change. They formulate the DJSSP as an MDP with the disjunctive graphs as the states and a set of PDRs as the action space.…”

Dynamic scheduling for dual-objective job shop with machine breakdown by reinforcement learning

“…The state of the manufacturing environment can be inaccurately and incompletely expressed due to artificial factors. Action space is mainly designed as priority rules 32,33 or parameter optimization, 34 which don't meet the desired execution efficiency in action exploration. In addition, some work even generalizes the model developed in the static environment to an uncertain resource environment, 35 lacking the learning process in the dynamic environment.…”

Dynamic Storage Location Assignment in Warehouses Using Deep Reinforcement Learning

“…Applied research often considers an additional dimension in the problem formulation inspired by real-world use-cases, such as stochasticity [13,14], machine flexibility [15][16][17], dynamic job releases [18], machine failures [19] or multi-objective optimization criteria [20,21]. These studies show the general feasibility of DRL to learn, but are typically not very competitive with expert systems.…”