<p><strong>Dynamic flexible job shop scheduling (DFJSS) has attracted a lot of attention from both academics and industries because of its widespread industrial impact in the real world. The main challenge in DFJSS involves simultaneous machine assignment and operation sequencing under dynamic environments such as new jobs arriving over time. Among the existing methods, scheduling heuristics stand out as effective approaches widely employed because of their simplicity and real-time responsiveness. Nevertheless, designing scheduling heuristics demands domain knowledge and is a time-consuming process. Genetic Programming (GP), as a hyper-heuristic method, has demonstrated notable success in automatically evolving scheduling heuristics for DFJSS. However, there is room for further enhancement in certain aspects to optimise GP's performance for DFJSS, particularly concerning diversity-based parent selection mechanisms, joint decision-making mechanisms, collaborative heuristic generation and selection by GP and reinforcement learning (RL), and multi-objective problem-solving capabilities.</strong></p><p>The overall goal of this thesis is to enhance the effectiveness of GP in automatically evolving high-quality scheduling heuristics for solving the DFJSS problems. Various machine learning and evolutionary computation techniques, such as lexicase selection, ensemble, semantic, RL, and multi-objective optimisation, are incorporated into GP to further improve the effectiveness of GP for DFJSS.</p><p>First, three novel diversity-based parent selection mechanisms are designed to enhance the GP's ability to explore the search space effectively, adapt to changes, and maintain a balance between exploration and exploitation for the successful evolution of high-quality scheduling heuristics. The three parent selection mechanisms are cluster selection, diverse partner selection, and lexicase selection. The new mechanisms consider not only the fitness of selected parents but also their diverse strengths, which is crucial for generating high-quality offspring. Experimental results demonstrate that the proposed methods, incorporating these newly designed diversity-based parent selection mechanisms, successfully enhance the quality of scheduling heuristics while promoting increased population diversity.</p><p>Second, this thesis proposes an innovative ensemble GP method that employs a population comprising both single individuals and ensembles, and allows the evolution of either a single scheduling heuristic for making individual decisions or a group of scheduling heuristics for making joint decisions for solving the DFJSS problem effectively. For creating an ensemble, an ensemble construction and selection strategy is developed for carefully choosing diverse and complementary individuals from the population. Additionally, new crossover and mutation operators are designed to generate offspring from the selected parents, which can be single individuals or ensembles. By maintaining a population that includes both single individuals and ensembles, the proposed method enables effective exploration of the search space through interbreeding. The experiment results show that the proposed method outperforms the compared algorithms including ensemble GP methods in terms of learned scheduling heuristics. Third, in addition to the widely used GP for automatic scheduling heuristics learning in DFJSS, RL has gained attention in recent years within this domain. This thesis proposes a niching GP-assisted RL method, taking advantage of both GP and RL to effectively address the DFJSS problems by intelligently heuristic generation by GP and selection by RL simultaneously. Specifically, instead of relying on manual scheduling heuristics, RL actions are replaced with scheduling heuristics evolved by the niching GP. RL is then employed to optimise and adapt these heuristics based on real-time feedback from the environment. The experiment results demonstrate the effectiveness of the proposed method in comparison to the widely used manual scheduling heuristics and the baseline RL method. This confirms the capability of GP to enhance RL's performance in addressing the DFJSS problem. Last, this thesis proposes two multi-objective GP methods for evolving a Pareto front of scheduling heuristics for solving the multi-objective DFJSS problem. The first method fills the gap by combining a well-known multi-objective evolutionary algorithm based on decomposition with GP, resulting in a novel multi-objective GP based on the decomposition method. This method successfully enhances the spreadability/diversity of the evolved Pareto front of scheduling heuristics. The second method enhances an existing state-of-the-art multi-objective GP approach (NSGPII) by incorporating semantic information. The experiment results demonstrate that NSGPII considering semantic diversity yields better performance compared with the original NSGPII. Moreover, NSGPII incorporating semantic similarity achieves even better performance, highlighting the importance of maintaining a reasonable semantic distance between offspring and their parents.</p>
