Adaptive reinforcement learning for task scheduling in aircraft maintenance

Abstract

This paper proposes using reinforcement learning (RL) to schedule maintenance tasks, which can significantly reduce direct operating costs for airlines. The approach consists of a static algorithm for long-term scheduling and an adaptive algorithm for rescheduling based on new maintenance information. To assess the performance of both approaches, three key performance indicators (KPIs) are defined: Ground Time, representing the hours an aircraft spends on the ground; Time Slack, measuring the proximity of tasks to their due dates; and Change Score, quantifying the similarity level between initial and adapted maintenance plans when new information surfaces. The results demonstrate the efficacy of RL in producing efficient maintenance plans, with the algorithms complementing each other to form a solid foundation for routine tasks and real-time responsiveness to new information. While the static algorithm performs slightly better in terms of Ground Time and Time Slack, the adaptive algorithm excels overwhelmingly in terms of Change Score, offering greater flexibility in handling new maintenance information. The proposed RL-based approach can improve the efficiency of aircraft maintenance and has the potential for further research in this area.

An innovative method to solve the maintenance task allocation and packing problem

“…The authors include the cost of predicted number of failures in addition to preventative maintenance costs, which some studies have overlooked. Silva et al . (2023) proposed the implementation of a maintenance scheduling framework using a static algorithm that produces the initial plan and, an adaptive algorithm that utilizes the reinforcement learning mechanism to update the plan.…”

“…(2012), operational profile, also used by Muchiri and Klaas (2009), Senturk and Ozkol (2018), Lee et al. (2022), Silva et al . (2023), resources available and zone limits, that are covered indirectly by Si et al.…”

“…They organize the task or tasks (clustering) using the single-task oriented approach, considering the resources available, task limits and the time when the aircraft is on the ground. Although presenting greater administrative complexity, the single-task oriented concept is also adopted by Senturk and Ozkol (2018), Silva et al (2023). Li et al (2015) research focused on testing the performance of a novel simplex algorithm in optimizing maintenance costs.…”

“…The authors include the cost of predicted number of failures in addition to preventative maintenance costs, which some studies have overlooked. Silva et al (2023) proposed the implementation of a maintenance scheduling framework using a static algorithm that produces the initial plan and, an adaptive algorithm that utilizes the reinforcement learning mechanism to update the plan. The optimization utilizes three key performance indicators: time slack, measure of the time distance from the due date, also considered by Muchiri and Klass (2009) and Senturk and Ozkol (2018), ground time, that is the period that the aircraft is not flying, and change score, reflecting the extent of changes need.…”