Sub‐second photon dose prediction via transformer neural networks

Abstract

Background: Fast dose calculation is critical for online and real-time adaptive therapy workflows. While modern physics-based dose algorithms must compromise accuracy to achieve low computation times, deep learning models can potentially perform dose prediction tasks with both high fidelity and speed. Purpose: We present a deep learning algorithm that, exploiting synergies between transformer and convolutional layers, accurately predicts broad photon beam dose distributions in few milliseconds. Methods: The proposed improved Dose Transformer Algorithm (iDoTA) maps arbitrary patient geometries and beam information (in the form of a 3D projected shape resulting from a simple ray tracing calculation) to their corresponding 3D dose distribution. Treating the 3D CT input and dose output volumes as a sequence of 2D slices along the direction of the photon beam, iDoTA solves the dose prediction task as sequence modeling. The proposed model combines a Transformer backbone routing long-range information between all elements in the sequence, with a series of 3D convolutions extracting local features of the data. We train iDoTA on a dataset of 1700 beam dose distributions, using 11 clinical volumetric modulated arc therapy (VMAT) plans (from prostate, lung, and head and neck cancer patients with 194-354 beams per plan) to assess its accuracy and speed. Results: iDoTA predicts individual photon beams in ≈ 50 ms with a high gamma pass rate of 97.72 ± 1.93% (2 mm, 2%). Furthermore, estimating full VMAT dose distributions in 6-12 s, iDoTA achieves state-of -the-art performance with a 99.51 ± 0.66% (2 mm, 2%) pass rate and an average relative dose error of 0.75 ± 0.36%. Conclusions: Offering the millisecond speed prediction per beam angle needed in online and real-time adaptive treatments, iDoTA represents a new state of the art in data-driven photon dose calculation. The proposed model can massively speed-up current photon workflows, reducing calculation times from few minutes to just a few seconds.

A Comparative Study of Deep Learning Dose Prediction Models for Cervical Cancer Volumetric Modulated Arc Therapy

“…Transformers-based dose prediction models have been proposed in recent years. 12 , 25 , 26 As a kind of typical transformers-based DL model, UNETR has never been used in dose prediction. 27 Research from Osman et al 28 showed that U-Net, attention U-Net, residual U-Net, and attention Res U-Net for H&N plans from OpenKBP-Grand Challenge data 7 had an almost comparable performance for voxel-wise dose prediction.…”

A probabilistic deep learning model of inter-fraction anatomical variations in radiotherapy

“…DAM’s main application in robust treatment planning and robust evaluation against inter-fraction movements involves sampling patient anatomies and calculating the corresponding dose distributions. With prediction times of few milliseconds per generated anatomy, DAM offers huge speed-up possibilities for plan evaluation when coupled to fast dose calculation algorithms (Perkó et al 2016 , Pastor-Serrano and Perkó 2022a , 2022b , Pastor-Serrano et al 2022 ). Few (3–5) representative scenarios corresponding to points around mean of the posterior distribution can be sampled to be used for scenario based robust optimization, which may translate into a dosimetric advantage or be used for margin reduction.…”

Application and progress of artificial intelligence in radiation therapy dose prediction

“…Currently, it has been applied in various domains such as image classification, target detection, and semantic segmentation, etc. Due to the lack of global feature acquisition in CNN, researchers have employed the transformer and developed several superior dose prediction models in combination with other DL models [74] , [75] , [76] , [77] , [78] , [79] , [80] , [81] . Wen et al [80] argue that existing DL models overlook the isodose lines and gradient information in dose maps.…”

“… For most isodose volumes,DSC > 0.91. Yue et al [75] /2022 Nasopharyngeal carcinoma:161 Training set:130 Validation set:11 Test set:20 3D U-Net Distance map Dose map The predicted dose error and DVH error are 7.51 % and 11.6 % lower than the mask-based method Jiao et al [76] /2023 Rectal cancer:120 Cervical cancer:42 Training set:116 Validation set:12 Testing set:34 Super-pixel-level GCN CT images Dose map HI 0.352;ΔD 95 0.150; ΔD mean 2.40E-2; ΔD max 1.68E-2 Pastor-Serrano et al [77] /2022 Training set:17 with disease sites of brain, head neck, lung, abdomen and pelvis Validation set:10 % of training set’s CT slices Test set:584 beam dose distributions iDoTA CT images Dose distribution Gamma pass rate in 50 ms:97.72 ± 1.93 %. Pass rate in 6–12 s:99.51 ± 0.66 %, average relative dose error:0.75 ± 0.36 %.…”