Automatic liver tumor localization using deep learning‐based liver boundary motion estimation and biomechanical modeling (DL‐Bio)

Shao, Hua‐Chieh; Huang, Xiaokun; Folkert, Michael R.; Wang, Jing; Zhang, You

doi:10.1002/mp.15275

Cited by 10 publications

(12 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each patient had a contrast-enhanced 4D-CT scan with liver tumors contoured by experienced physicians. The details of the dataset have been presented previously (Shao et al 2021), thus only some key characteristics were summarized in table 1.…”

Section: Dataset Curation Augmentation and Model Trainingmentioning

confidence: 99%

Real-time liver tumor localization via combined surface imaging and a single x-ray projection

Shao

Wang

et al. 2023

Phys. Med. Biol.

Self Cite

View full text Add to dashboard Cite

Objective: Real-time imaging, a building block of real-time adaptive radiotherapy, provides instantaneous knowledge of anatomical motion to drive delivery adaptation to improve patient safety and treatment efficacy. The temporal constraint of real-time imaging (<500 milliseconds) significantly limits the imaging signals that can be acquired, rendering volumetric imaging and 3D tumor localization extremely challenging. Real-time liver imaging is particularly difficult, compounded by the low soft tissue contrast within the liver. We proposed a deep learning (DL)-based framework (Surf-X-Bio), to track 3D liver tumor motion in real-time from combined optical surface image and a single on-board x-ray projection. Approach: Surf-X-Bio performs mesh-based deformable registration to track/localize liver tumors volumetrically via three steps. First, a DL model was built to estimate liver boundary motion from an optical surface image, using learnt motion correlations between the respiratory-induced external body surface and liver boundary. Second, the residual liver boundary motion estimation error was further corrected by a graph neural network (GNN)-based DL model, using information extracted from a single x-ray projection. Finally, a biomechanical modeling-driven DL model was applied to solve the intra-liver motion for tumor localization, using the liver boundary motion derived via prior steps. Main results: Surf-X-Bio demonstrated higher accuracy and better robustness in tumor localization, as compared to surface-image-only and x-ray-only models. By Surf-X-Bio, the mean (±s.d.) 95-percentile Hausdorff distance of the liver boundary from the ‘ground-truth’ decreased from 9.8 (±4.5) mm (before motion estimation) to 2.4 (±1.6) mm. The mean (± s.d.) center-of-mass localization error of the liver tumors decreased from 8.3 (±4.8) mm to 1.9 (±1.6) mm. Significance: Surf-X-Bio can accurately track liver tumors from combined surface imaging and x-ray imaging. The fast computational speed (<250 milliseconds per inference) allows it to be applied clinically for real-time motion management and adaptive radiotherapy.

show abstract

Section: Dataset Curation Augmentation and Model Trainingmentioning

confidence: 99%

Real-time liver tumor localization via combined surface imaging and a single x-ray projection

Shao

Wang

et al. 2023

Phys. Med. Biol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Such knowledge potentially allows the treatment to adapt to real-time changes to improve tumor targeting accuracy (Keall et al 2018, Booth et al 2021. Due to the stringent temporal resolution requirement (Keall et al 2018, Skouboe et al 2019, Keall et al 2021 of real-time imaging, the target volumetric information will be severely under-sampled via cone-beam computed tomography (CBCT) imaging, liver tumors to within 2 mm from as few as 20 projections (Shao et al 2021), it requires the projections to be sparsely distributed over a full scan angle and cannot achieve real-time imaging yet.…”

Section: Introductionmentioning

confidence: 99%

Real-time liver tumor localization via a single x-ray projection using deep graph neural network-assisted biomechanical modeling

et al. 2022

Self Cite

View full text Add to dashboard Cite

Objective. Real-time imaging is highly desirable in image-guided radiotherapy, as it provides instantaneous knowledge of patients’ anatomy and motion during treatments and enables online treatment adaptation to achieve the highest tumor targeting accuracy. Due to extremely limited acquisition time, only one or few x-ray projections can be acquired for real-time imaging, which poses a substantial challenge to localize the tumor from the scarce projections. For liver radiotherapy, such a challenge is further exacerbated by the diminished contrast between the tumor and the surrounding normal liver tissues. Here, we propose a framework combining graph neural network-based deep learning and biomechanical modeling to track liver tumor in real-time from a single onboard x-ray projection. Approach. Liver tumor tracking is achieved in two steps. First, a deep learning network is developed to predict the liver surface deformation using image features learned from the x-ray projection. Second, the intra-liver deformation is estimated through biomechanical modeling, using the liver surface deformation as the boundary condition to solve tumor motion by finite element analysis. The accuracy of the proposed framework was evaluated using a dataset of 10 patients with liver cancer. Main results. The results show accurate liver surface registration from the graph neural network-based deep learning model, which translates into accurate, fiducial-less liver tumor localization after biomechanical modeling (<1.2 (±1.2) mm average localization error). Significance. The method demonstrates its potentiality towards intra-treatment and real-time 3D liver tumor monitoring and localization. It could be applied to facilitate 4D dose accumulation, multi-leaf collimator tracking and real-time plan adaptation. The method can be adapted to other anatomical sites as well.

show abstract

“…The key procedures were summarized below: (i) For each patient, a deformable image registration was performed on the 4D‐CT via Elastix, 41 using the 0% phase as the reference (moving) phase, to obtain the volumetric DVFs between the reference phase and the other phases (10%−90%). The registration accuracy at the liver boundaries was enhanced using the liver‐density overwrite technique to improve the image contrast at the boundaries 32 . (ii) A patient‐specific motion model was created by performing PCA on the resulting DVFs.…”

Section: Methodsmentioning

confidence: 99%

“…The registration accuracy at the liver boundaries was enhanced using the liver-density overwrite technique to improve the image contrast at the boundaries. 32 (ii) A patient-specific motion model was created by performing PCA on the resulting DVFs.…”

Section: Dataset Curation and Augmentationmentioning

confidence: 99%

“…To address these challenges, we proposed a DLbased, deformable registration-driven framework, X360, to solve deformable organ boundary motion at arbitrary X-ray projection angles. Following our previous line of work, 29,30,32 we applied X360 to the site of the liver, the low imaging contrast of which greatly challenges the current X-ray imaging and motion estimation techniques. Via X360, the liver boundary was represented as a surface mesh, which provides enhanced flexibility in defining local and global motion at appropriate resolutions through varying mesh element sizes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Real‐time liver motion estimation via deep learning‐based angle‐agnostic X‐ray imaging

Shao,

Li,

Wang

et al. 2023

Medical Physics

View full text Add to dashboard Cite

BackgroundReal‐time liver imaging is challenged by the short imaging time (within hundreds of milliseconds) to meet the temporal constraint posted by rapid patient breathing, resulting in extreme under‐sampling for desired 3D imaging. Deep learning (DL)‐based real‐time imaging/motion estimation techniques are emerging as promising solutions, which can use a single X‐ray projection to estimate 3D moving liver volumes by solved deformable motion. However, such techniques were mostly developed for a specific, fixed X‐ray projection angle, thereby impractical to verify and guide arc‐based radiotherapy with continuous gantry rotation.PurposeTo enable deformable motion estimation and 3D liver imaging from individual X‐ray projections acquired at arbitrary X‐ray scan angles, and to further improve the accuracy of single X‐ray‐driven motion estimation.MethodsWe developed a DL‐based method, X360, to estimate the deformable motion of the liver boundary using an X‐ray projection acquired at an arbitrary gantry angle (angle‐agnostic). X360 incorporated patient‐specific prior information from planning 4D‐CTs to address the under‐sampling issue, and adopted a deformation‐driven approach to deform a prior liver surface mesh to new meshes that reflect real‐time motion. The liver mesh motion is solved via motion‐related image features encoded in the arbitrary‐angle X‐ray projection, and through a sequential combination of rigid and deformable registration modules. To achieve the angle agnosticism, a geometry‐informed X‐ray feature pooling layer was developed to allow X360 to extract angle‐dependent image features for motion estimation. As a liver boundary motion solver, X360 was also combined with priorly‐developed, DL‐based optical surface imaging and biomechanical modeling techniques for intra‐liver motion estimation and tumor localization.ResultsWith geometry‐aware feature pooling, X360 can solve the liver boundary motion from an arbitrary‐angle X‐ray projection. Evaluated on a set of 10 liver patient cases, the mean (± s.d.) 95‐percentile Hausdorff distance between the solved liver boundary and the “ground‐truth” decreased from 10.9 (±4.5) mm (before motion estimation) to 5.5 (±1.9) mm (X360). When X360 was further integrated with surface imaging and biomechanical modeling for liver tumor localization, the mean (± s.d.) center‐of‐mass localization error of the liver tumors decreased from 9.4 (± 5.1) mm to 2.2 (± 1.7) mm.ConclusionX360 can achieve fast and robust liver boundary motion estimation from arbitrary‐angle X‐ray projections for real‐time imaging guidance. Serving as a surface motion solver, X360 can be integrated into a combined framework to achieve accurate, real‐time, and marker‐less liver tumor localization.

show abstract

Automatic liver tumor localization using deep learning‐based liver boundary motion estimation and biomechanical modeling (DL‐Bio)

Cited by 10 publications

References 80 publications

Real-time liver tumor localization via combined surface imaging and a single x-ray projection

Real-time liver tumor localization via combined surface imaging and a single x-ray projection

Real-time liver tumor localization via a single x-ray projection using deep graph neural network-assisted biomechanical modeling

Real‐time liver motion estimation via deep learning‐based angle‐agnostic X‐ray imaging

Contact Info

Product

Resources

About