OpenKBP-Opt: an international and reproducible evaluation of 76 knowledge-based planning pipelines

Abstract

Objective. To establish an open framework for developing plan optimization models for knowledge-based planning (KBP). Approach. Our framework includes radiotherapy treatment data (i.e. reference plans) for 100 patients with head-and-neck cancer who were treated with intensity-modulated radiotherapy. That data also includes high-quality dose predictions from 19 KBP models that were developed by different research groups using out-of-sample data during the OpenKBP Grand Challenge. The dose predictions were input to four fluence-based dose mimicking models to form 76 unique KBP pipelines that generated 7600 plans (76 pipelines × 100 patients). The predictions and KBP-generated plans were compared to the reference plans via: the dose score, which is the average mean absolute voxel-by-voxel difference in dose; the deviation in dose-volume histogram (DVH) points; and the frequency of clinical planning criteria satisfaction. We also performed a theoretical investigation to justify our dose mimicking models. Main results. The range in rank order correlation of the dose score between predictions and their KBP pipelines was 0.50–0.62, which indicates that the quality of the predictions was generally positively correlated with the quality of the plans. Additionally, compared to the input predictions, the KBP-generated plans performed significantly better (P < 0.05; one-sided Wilcoxon test) on 18 of 23 DVH points. Similarly, each optimization model generated plans that satisfied a higher percentage of criteria than the reference plans, which satisfied 3.5% more criteria than the set of all dose predictions. Lastly, our theoretical investigation demonstrated that the dose mimicking models generated plans that are also optimal for an inverse planning model. Significance. This was the largest international effort to date for evaluating the combination of KBP prediction and optimization models. We found that the best performing models significantly outperformed the reference dose and dose predictions. In the interest of reproducibility, our data and code is freely available.

3D dose prediction for Gamma Knife radiosurgery using deep learning and data modification

“…The lower standard deviation on the gamma passing rates of GAN-GK and U-Net-GK predictions also indicate greater consistency. Since better dose predictions are more likely to lead to higher quality plans [11], the presented prediction methodology would serve well as the first stage of a two-stage GKRS KBP pipeline.…”

“…In this paper, we develop a novel GKRS dose prediction approach. This is an important first step towards creating an automated GKRS planning pipeline since the quality of plans produced by such a pipeline is positively correlated with the quality of the dose predictions [11]. Our approach accommodates any size, number, and shape of targets without compromising the spatial resolution of the predicted dose.…”

A novel mathematical model to generate semi‐automated optimal IMRT treatment plan based on predicted 3D dose distribution and prescribed dose

“…The best result for each ROI is displayed in green bold. In the case of solving time, the average of the QuadLin equals 341 s which is less than three models of the recent study by Babier et al 35 In addition, it has been shown that the QuadLin model has the potential to reduce the solving time even more, while the quality of the solution remains at a desirable level. 45,46 At last, equal weighting of the ROIs and manual tuning of the weights of the objective function are some limitations of the current research which are the areas of future research.…”

“…The best result for each ROI is displayed in green bold. In the case of solving time, the average of the QuadLin equals 341 s which is less than three models of the recent study by Babier et al 35 . In addition, it has been shown that the QuadLin model has the potential to reduce the solving time even more, while the quality of the solution remains at a desirable level 45,46 …”

“…Furthermore, recent studies [28][29][30][31][32][33][34] (with similar research methods) suffer from one or more weaknesses such as a lack of statistical tests to support the results' significance; a shortage of discussions on the solving time and the computation efficiency; debat-able results because of the small type of tumor (having few ROIs). Here we compare the results of the QuadLin model with four models presented by Babier et al 35 titled by MaxAbs, MeanAbs, MaxRel, and MeanRel. Comparison of all models is based on the results of 30 randomly selected patients (see II-B).…”