Large-State Reinforcement Learning for Hyper-Heuristics

Abstract

Hyper-heuristics are a domain-independent problem solving approach where the main task is to select effective chains of problem-specific low-level heuristics on the fly for an unseen instance. This task can be seen as a reinforcement learning problem, however, the information available to the hyper-heuristic is very limited, usually leading to very limited state representations. In this work, for the first time we use the trajectory of solution changes for a larger set of features for reinforcement learning in the novel hyper-heuristic LAST-RL (Large-State Reinforcement Learning). Further, we introduce a probability distribution for the exploration case in our epsilon-greedy policy that is based on the idea of Iterated Local Search to increase the chance to sample good chains of low-level heuristics. The benefit of the collaboration of our novel components is shown on the academic benchmark of the Cross Domain Heuristic Challenge 2011 consisting of six different problem domains. Our approach can provide state-of-the-art results on this benchmark where it outperforms recent hyper-heuristics based on reinforcement learning, and also demonstrates high performance on a benchmark of complex real-life personnel scheduling domains.

Investigating Large Neighbourhood Search for Bus Driver Scheduling

“…In particular, the collective agreement has stringent rules requiring the drivers to take frequent breaks, with the eventual option of splitting them into multiple parts. This problem has been introduced recently in the literature, and to the best of our knowledge, the recently introduced exact approach based on Branch and Price (Kletzander, Musliu, and Van Hentenryck 2021), and meta-heuristic and hyperheuristic based approaches (Kletzander and Musliu 2020;Kletzander, Mazzoli, and Musliu 2022;Rosati et al 2023;Kletzander and Musliu 2023) represent the current state of the art for this problem. Although these approaches give very good results, exact methods are computationally too expensive for large real-life-based instances and heuristic methods cannot obtain optimal solutions.…”