3D–2D image registration for target localization in spine surgery: investigation of similarity metrics providing robustness to content mismatch

Silva, T. De; Uneri, Ali; Ketcha, Michael D.; Reaungamornrat, S.; Kleinszig, Gerhard; Vogt, Sebastian; Aygün, Nafi; Lo, Sheng Fu L.; Wolinsky, Jean Paul; Siewerdsen, Jeffrey H.

doi:10.1088/0031-9155/61/8/3009

Cited by 94 publications

(92 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The resulting transformation enables the vertebral labels in the CT image to be accurately projected and overlaid in p . The overall framework illustrated in figure 2 is consistent with that in Otake et al (2012, 2013, 2015) and De Silva et al (2016a).…”

Section: Methodssupporting

confidence: 73%

“…Similarity between the DRR and the intraoperative radiograph was evaluated using Gradient Orientation (GO), which was shown in (De Silva et al 2016a) to provide a high degree of robustness against image content mismatch (e.g., presence of surgical tools in the radiograph but not the CT) as well as poor radiographic image quality. The GO similarity was defined as:

G O = \frac{1}{max false(N, N_{L B} false)} true \sum_{{i : \nabla D R R_{i} > t \cap \nabla p_{i} > t}} w^{'} false(i false)

where w^{'} false(i false) = \frac{2 - \ln false(| θ_{i} | + 1 false)}{2}

and reflects the pixel-wise similarity in gradient direction, w ′, among pixels whose gradient magnitude passes a threshold t in both images, defined as the median gradient intensity.…”

Section: Methodsmentioning

confidence: 99%

“…Here, θ i is the angle difference (radians) in gradient direction between the DRR and p at pixel i . The normalization constant N is the number of pixel locations for which the metric is evaluated, and N LB is a lower bound cutoff set to 30% of the total number of pixels to penalize low counts associated with poor overlap (shown in previous studies to be a stable, nominal parameter setting for this application in De Silva et al [2016a]). …”

Section: Methodsmentioning

confidence: 99%

“…A wrong-level error can lead to suboptimal (or failed) surgical product and potentially costly litigation (Mody et al 2008). To prevent such occurrences and potentially improve workflow and confidence in target localization, a 3D-2D registration framework (called LevelCheck) has been shown to automatically overlay relevant target anatomy such as vertebral labels from 3D preoperative imaging (CT or MR) to the intraoperative 2D image as a means of decision support (Otake et al 2012, 2013, 2015, De Silva et al 2016a, b). LevelCheck has been shown to provide robust registration under many challenging scenarios that are commonly encountered in clinical images, including poor image quality and content mismatch (e.g., surgical instrumentation appearing in the radiograph that is not in the 3D image); however, the accuracy of rigid 3D-2D registration may be compromised in the presence of deformation.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Multi-stage 3D–2D registration for correction of anatomical deformation in image-guided spine surgery

et al. 2017

Self Cite

View full text Add to dashboard Cite

Purpose A multi-stage image-based 3D-2D registration method is presented that maps annotations in a 3D image (e.g., point labels annotating individual vertebrae in preoperative CT) to an intraoperative radiograph in which the patient has undergone non-rigid anatomical deformation due to changes in patient positioning or due to the intervention itself. Methods The proposed method (termed msLevelCheck) extends a previous rigid registration solution (LevelCheck) to provide an accurate mapping of vertebral labels in the presence of spinal deformation. The method employs a multi-stage series of rigid 3D-2D registrations performed on sets of automatically determined and increasingly localized sub-images, with the final stage achieving a rigid mapping for each label to yield a locally rigid yet globally deformable solution. The method was evaluated first in a phantom study in which a CT image of the spine was acquired followed by a series of 7 mobile radiographs with increasing degree of deformation applied. Second, the method was validated using a clinical data set of patients exhibiting strong spinal deformation during thoracolumbar spine surgery. Registration accuracy was assessed using projection distance error (PDE) and failure rate (PDE > 20 mm – i.e., label registered outside vertebra). Results The msLevelCheck method was able to register all vertebrae accurately for all cases of deformation in the phantom study, improving the maximum PDE of the rigid method from 22.4 mm to 3.9 mm. The clinical study demonstrated the feasibility of the approach in real patient data by accurately registering all vertebral labels in each case, eliminating all instances of failure encountered in the conventional rigid method. Conclusion The multi-stage approach demonstrated accurate mapping of vertebral labels in the presence of strong spinal deformation. The msLevelCheck method maintains other advantageous aspects of the original LevelCheck method (e.g., compatibility with standard clinical workflow, large capture range, and robustness against mismatch in image content) and extends capability to cases exhibiting strong changes in spinal curvature.

show abstract

Section: Methodssupporting

confidence: 73%

G O = \frac{1}{max false(N, N_{L B} false)} true \sum_{{i : \nabla D R R_{i} > t \cap \nabla p_{i} > t}} w^{'} false(i false)

where w^{'} false(i false) = \frac{2 - \ln false(| θ_{i} | + 1 false)}{2}

and reflects the pixel-wise similarity in gradient direction, w ′, among pixels whose gradient magnitude passes a threshold t in both images, defined as the median gradient intensity.…”

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-stage 3D–2D registration for correction of anatomical deformation in image-guided spine surgery

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…It directly estimates the 3D prostate position by matching the projection positions of the fiducial markers with the measured ones. This idea is motivated by recent advances in 2D-3D image registration problems, where motion in the 3D space can be accurately determined by 2D projection images based on projection geometry via optimization approaches (Otake et al , 2015; Uneri et al , 2015; De Silva et al , 2016). Since a single projection image cannot accurately determine the 3D prostate position because of missing geometric information along the direction of the x-ray projection, we assume a temporal correlation between prostate positions at different moments.…”

Section: Introductionmentioning

confidence: 99%

A new method to reconstruct intra-fractional prostate motion in volumetric modulated arc therapy

et al. 2017

View full text Add to dashboard Cite

Intra-fractional motion is a concern during prostate radiation therapy, as it may cause deviations between planned and delivered radiation doses. Because accurate motion information during treatment delivery is critical to address dose deviation, we developed the Projection Marker Matching Method (PM3), a novel method for prostate motion reconstruction in volumetric modulated arc therapy. The purpose of this method is to reconstruct in-treatment prostate motion trajectory using projected positions of implanted fiducial markers measured in kV x-ray projection images acquired during treatment delivery. We formulated this task as a quadratic optimization problem. The objective function penalized the distance from the reconstructed 3D position of each fiducial marker to the corresponding straight line, defined by the x-ray projection of the marker. Rigid translational motion of the prostate and motion smoothness along the temporal dimension were assumed and incorporated into the optimization model. We tested the motion reconstruction method in both simulation and phantom experimental studies. We quantified the accuracy using 3D normalized root-mean-square (RMS) error defined as the norm of a vector containing ratios between the absolute RMS errors and corresponding motion ranges in three dimensions. In the simulation study with realistic prostate motion trajectories, the 3D normalized RMS error was on average 0.164 (range from 0.097 to 0.333). In an experimental study, a prostate phantom was driven to move along a realistic prostate motion trajectory. The 3D normalized RMS error was 0.172. We also examined the impact of the model parameters on reconstruction accuracy, and found that a single set of parameters can be used for all the tested cases to accurately reconstruct the motion trajectories. The motion trajectory derived by PM3 may be incorporated into novel strategies, including 4D dose reconstruction and adaptive treatment replanning to address motion-induced dose deviation.

show abstract

Deformable registration of lateral cephalogram and cone‐beam computed tomography image

Zhang

Qin

et al. 2021

Medical Physics

View full text Add to dashboard Cite

Purpose: This study aimed to design and evaluate a novel method for the registration of 2D lateral cephalograms and 3D craniofacial cone‐beam computed tomography (CBCT) images, providing patient‐specific 3D structures from a 2D lateral cephalogram without additional radiation exposure. Methods: We developed a cross‐modal deformable registration model based on a deep convolutional neural network. Our approach took advantage of a low‐dimensional deformation field encoding and an iterative feedback scheme to infer coarse‐to‐fine volumetric deformations. In particular, we constructed a statistical subspace of deformation fields and parameterized the nonlinear mapping function from an image pair, consisting of the target 2D lateral cephalogram and the reference volumetric CBCT, to a latent encoding of the deformation field. Instead of the one‐shot registration by the learned mapping function, a feedback scheme was introduced to progressively update the reference volumetric image and to infer coarse‐to‐fine deformations fields, accounting for the shape variations of anatomical structures. A total of 220 clinically obtained CBCTs were used to train and validate the proposed model, among which 120 CBCTs were used to generate a training dataset with 24k paired synthetic lateral cephalograms and CBCTs. The proposed approach was evaluated on the deformable 2D–3D registration of clinically obtained lateral cephalograms and CBCTs from growing and adult orthodontic patients. Results: Strong structural consistencies were observed between the deformed CBCT and the target lateral cephalogram in all criteria. The proposed method achieved state‐of‐the‐art performances with the mean contour deviation of 0.41±0.12 mm on the anterior cranial base, 0.48±0.17 mm on the mandible, and 0.35±0.08 mm on the maxilla, respectively. The mean surface mesh ranged from 0.78 to 0.97 mm on various craniofacial structures, and the LREs ranged from 0.83 to 1.24 mm on the growing datasets regarding 14 landmarks. The proposed iterative feedback scheme handled the structural details and improved the registration. The resultant deformed volumetric image was consistent with the target lateral cephalogram in both 2D projective planes and 3D volumetric space regarding the multicategory craniofacial structures. Conclusions: The results suggest that the deep learning‐based 2D–3D registration model enables the deformable alignment of 2D lateral cephalograms and CBCTs and estimates patient‐specific 3D craniofacial structures.

show abstract

3D–2D image registration for target localization in spine surgery: investigation of similarity metrics providing robustness to content mismatch

Cited by 94 publications

References 27 publications

Multi-stage 3D–2D registration for correction of anatomical deformation in image-guided spine surgery

Multi-stage 3D–2D registration for correction of anatomical deformation in image-guided spine surgery

A new method to reconstruct intra-fractional prostate motion in volumetric modulated arc therapy

Deformable registration of lateral cephalogram and cone‐beam computed tomography image

Contact Info

Product

Resources

About