Patient-specific quality assurance failure prediction with deep tabular models

Abstract

Purpose: Patient-specific quality assurance (PSQA) failures in radiotherapy can cause a delay in patient care and increase the workload and stress of staff. We developed a tabular transformer model based directly on the multi-leaf collimator (MLC) leaf positions (without any feature engineering) to predict IMRT PSQA failure in advance. This neural model provides an end-to-end differentiable map from MLC leaf positions to the probability of PSQA plan failure, which could be useful for regularizing gradient-based leaf sequencing optimization algorithms and generating a plan that is more likely to pass PSQA.
Method: We retrospectively collected DICOM RT PLAN files of 968 patient plans treated with volumetric arc therapy. We constructed a beam-level tabular dataset with 1873 beams as samples and MLC leaf positions as features. We trained an attention-based neural network FT-Transformer to predict the ArcCheck-based PSQA gamma pass rates. In addition to the regression task, we evaluated the model in the binary classification context predicting the pass or fail of PSQA. The performance was compared to the results of the two leading tree ensemble methods (CatBoost and XGBoost) and a non-learned method based on mean-MLC-gap.
Results: The FT-Transformer model achieves 1.44% Mean Absolute Error (MAE) in the regression task of the gamma pass rate prediction and performs on par with XGBoost (1.53% MAE) and CatBoost (1.40% MAE). In the binary classification task of PSQA failure prediction, FT-Transformer achieves 0.85 ROC AUC (compared to the mean-MLC-gap complexity metric achieving 0.72 ROC AUC). Moreover, FT-Transformer, CatBoost, and XGBoost all achieve 80% true positive rate while keeping the false positive rate under 20%.
Conclusions: We demonstrated that reliable PSQA failure predictors can be successfully developed based solely on MLC leaf positions. FT-Transformer offers an unprecedented benefit of providing an end-to-end differentiable map from MLC leaf positions to the probability of PSQA failure.