Can the Student Outperform the Master? A Plan Comparison Between Pinnacle Auto-Planning and Eclipse knowledge-Based RapidPlan Following a Prostate-Bed Plan Competition

Abstract

Purpose: Pinnacle Auto-Planning and Eclipse RapidPlan are 2 major commercial automated planning engines that are fundamentally different: Auto-Planning mimics real planners in the iterative optimization, while RapidPlan generates static dose objectives from estimations predicted based on a prior knowledge base. This study objectively compared their performances on intensity-modulated radiotherapy planning for prostate fossa and lymphatics adopting the plan quality metric used in the 2011 American Association of Medical Dosimetrists Plan Challenge. Methods: All plans used an identical intensity-modulated radiotherapy beam setup and a simultaneous integrated boost prescription (68 Gy/56 Gy to prostate fossa/lymphatics). Auto-Planning was used to retrospectively plan on 20 patients, which were subsequently employed as the library to build an RapidPlan model. To compare the 2 engines’ performances, a test set including 10 patients and the Plan Challenge patient was planned by both Auto-Planning (master) and RapidPlan (student) without manual intervention except for a common dose normalization and evaluated using the plan quality metric that included 14 quantitative submetrics ranging over target coverage, spillage, and organ at risk doses. Plan quality metric scores were compared between the Auto-Planning and RapidPlan plans using the Mann-Whitney U test. Results: There was no significant difference between the overall performance of the 2 engines on the 11 test cases ( P = .509). Among the 14 submetrics, Auto-Planning and RapidPlan showed no significant difference on most submetrics except for 2. On the Plan Challenge case, Auto-Planning scored 129.9 and RapidPlan scored 130.3 out of 150, as compared with the average score of 116.9 ± 16.4 (range: 58.2-142.5) among the 125 Plan Challenge participants. Conclusion: Using an innovative study design, an objective comparison has been conducted between 2 major commercial automated inverse planning engines. The 2 engines performed comparably with each other and both yielded plans at par with average human planners. Using a constant-performing planner (Auto-Planning) to train and to compare, RapidPlan was found to yield plans no better than but as good as its library plans.

Knowledge‐based radiation treatment planning: A data‐driven method survey

“…The cumulative publication curve (Figure 1b) demonstrates an increased rate in the number of traditional KBP‐based articles between 2015 and 2018. Between 2018 and 2020, most studies on traditional KBP methods have been based on a commercial KBP module rather than further development of earlier ML or statistical methods 33–45 . This is certainly not because traditional KBP methods have been fully explored, but presumably due to the wider availability of a commercial KBP module in clinical practice and increased interest in exploring DL methods owing to their recent success in achieving state‐of‐the‐art performance in many medical applications.…”

“…Between 2018 and 2020, most studies on traditional KBP methods have been based on a commercial KBP module rather than further development of earlier ML or statistical methods. 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 This is certainly not because traditional KBP methods have been fully explored, but presumably due to the wider availability of a commercial KBP module in clinical practice and increased interest in exploring DL methods owing to their recent success in achieving state‐of‐the‐art performance in many medical applications. In recent years, a wide array of research on developing new prediction models has transitioned from traditional KBP methods to DL KBP methods.…”

Assessment of efficacy in automated plan generation for Varian Ethos intelligent optimization engine

“…There has been a significant improvement in radiation therapy treatment planning, optimization, and delivery in the last decade 3–5 . Several techniques have been developed to optimize overall treatment planning and delivery to increase therapeutic gain 4–15 . Lately, online adaptive therapy with daily treatment planning and delivery is being introduced into the radiation therapy community 16–20 .…”

“…Both techniques have shown to create acceptable plans while minimizing planner variability. Pinnacle (Fitchburg, Wisconsin, USA) auto‐planning (AP) is an example of the first technique and Varian (Palo Alto, California, USA) Eclipse RapidPlan (RP) is an example of the second technique 6,7 . Several groups have investigated Pinnacle AP and Eclipse RP for different anatomical sites 6–13 .…”

Evaluation of a generalized knowledge-based planning performance for VMAT irradiation of breast and locoregional lymph nodes—Internal mammary and/or supraclavicular regions

“…Automatic planning systems were recently developed, so that the inter-operator variability could be reduced, the planning time could be spared, and the plan’s quality could possibly be improved [ 10 – 16 ]. The Knowledge based (KB) optimization approach, is a good alternative to the “automatic planning” because of its capability to improve both the plan consistency and the planning efficiency [ 17 , 18 ].…”