Dynamic cone-beam CT reconstruction using spatial and temporal implicit neural representation learning (STINR)

Abstract

Objective: Dynamic cone-beam CT (CBCT) imaging is highly desired in image-guided radiation therapy to provide volumetric images with high spatial and temporal resolutions to enable applications including tumor motion tracking/prediction and intra-delivery dose calculation/accumulation. However, the dynamic CBCT reconstruction is a substantially challenging spatiotemporal inverse problem, due to the extremely limited projection sample available for each CBCT reconstruction (one projection for one CBCT volume). Approach: We developed a simultaneous spatial and temporal implicit neural representation (STINR) method for dynamic CBCT reconstruction. STINR mapped the unknown image and the evolution of its motion into spatial and temporal multi-layer perceptrons (MLPs), and iteratively optimized the neuron weighting of the MLPs via acquired projections to represent the dynamic CBCT series. In addition to the MLPs, we also introduced prior knowledge, in the form of principal component analysis (PCA)-based patient-specific motion models, to reduce the complexity of the temporal INRs to address the ill-conditioned dynamic CBCT reconstruction problem. We used the extended-cardiac-torso (XCAT) phantom and a patient 4D-CBCT dataset to simulate different lung motion scenarios to evaluate STINR. The scenarios contain motion variations including motion baseline shifts, motion amplitude/frequency variations, and motion non-periodicity. The XCAT scenarios also contain inter-scan anatomical variations including tumor shrinkage and tumor position change. Main results: STINR shows consistently higher image reconstruction and motion tracking accuracy than a traditional PCA-based method and a polynomial-fitting based neural representation method. STINR tracks the lung target to an averaged center-of-mass error of 1-2 mm, with corresponding relative errors of reconstructed dynamic CBCTs around 10%. Significance: STINR offers a general framework allowing accurate dynamic CBCT reconstruction for image-guided radiotherapy. It is a one-shot learning method that does not rely on pre-training and is not susceptible to generalizability issues. It also allows natural super-resolution. It can be readily applied to other imaging modalities as well.

Dynamic CBCT imaging using prior model-free spatiotemporal implicit neural representation (PMF-STINR)

“…The erroneous PCA motion model significantly impacted the referenceframe CBCT quality for STINR PCA-4DCBCT . In comparison, using the PCA motion model derived from a simulated artifact-free 4D-CT, STINR PCA-4DCT generated better results than STINR PCA-4DCBCT , with an accuracy level on par with that reported in the original STINR study (Zhang et al 2023). However, the highquality, artifacts-free 4D-CT of XCAT is difficult to realize in real clinical cases.…”

3D cine-magnetic resonance imaging using spatial and temporal implicit neural representation learning (STINR-MR)

“…To regularize the solution of motion, we incorporated a principal component analysis (PCA)-based patient-specific motion model into the framework (top box of figure 1 ). PCA-based motion model introduced prior motion modes to significantly reduce the dimensionality of the unknown DVFs (Zhang et al 2013 , Zhang et al 2019 , Zhang et al 2023 ). To derive the PCA-based motion model, a previously acquired, motion-binned 4D-MRI can be used.…”

“…To address this challenge, we initialized the spatial INR prior to the joint training (i.e. a warm start for the joint training), and designed a STINR-MR training scheme with progressively added complexity to avoid the local minimum (Zhang et al 2023 ). The training scheme contained three stages with designated loss functions.…”

“…With these advantages, INR has been applied to solve x-ray-based and MR-based reconstruction problems from sparse-view measurements (Shen et al 2022 , Zha et al 2022 ). Furthermore, INR-based reconstruction algorithms for dynamic computed tomography (CT) and cone-beam CT were also developed (Reed et al 2021 , Zhang et al 2023 ).…”

“…Inspired by our recent work in INR-based cone-beam CT reconstruction (Zhang et al 2023 ), in this work we proposed a joint reconstruction and deformable registration-based framework using spatial and temporal INRs for dynamic 3D cine-MRI reconstruction (STINR-MR). STINR-MR uses spatiotemporal INRs to learn, reconstruct, and map 3D cine-MRI volumes and the corresponding time-varying motion.…”