Variability in commercially available deformable image registration: A multi‐institution analysis using virtual head and neck phantoms

Abstract

Purpose: The purpose of this study was to evaluate the performance of three common deformable image registration (DIR) packages across algorithms and institutions. Methods and Materials:The Deformable Image Registration Evaluation Project (DIREP) provides ten virtual phantoms derived from computed tomography (CT) datasets of head-and-neck cancer patients over a single treatment course. Using the DIREP phantoms, DIR results from 35 institutions were submitted using either Velocity, MIM, or Eclipse. Submitted deformation vector fields (DVFs) were compared to ground-truth DVFs to calculate target registration error (TRE) for six regions of interest (ROIs). Statistical analysis was performed to determine the variability between each DIR software package and the variability of users within each algorithm.Results: Overall mean TRE was 2.04 ± 0.35 mm for Velocity, 1.10 ± 0.29 mm for MIM, and 2.35 ± 0.15 mm for Eclipse. The MIM mean TRE was significantly different than both Velocity and Eclipse for all ROIs. Velocity and Eclipse mean TREs were not significantly different except for when evaluating the registration of the cord or mandible. Significant differences between institutions were found for the MIM and Velocity platforms. However, these differences could be explained by variations in Velocity DIR parameters and MIM software versions.Conclusions: Average TRE was shown to be <3 mm for all three software platforms. However, maximum errors could be larger than 2 cm indicating that care should be exercised when using DIR. While MIM performed statistically better than the other packages, all evaluated algorithms had an average TRE better than the largest voxel dimension. For the phantoms studied here, significant differences between algorithm users were minimal suggesting that the algorithm used may have more impact on DIR accuracy than the particular registration technique employed. A significant difference in TRE was discovered between MIM versions showing that DIR QA should be performed after software upgrades as recommended by TG-132.

Evaluation and risk factors of volume and dose differences of selected structures in patients with head and neck cancer treated on Helical TomoTherapy by using Deformable Image Registration tool

“…The results showed the reasonable accuracy potential to be used in clinical settings. 7, [39][40][41] Our B-ART criteria are very promising. Validation of the PG volume alterations, CTV, and PTV coverage was also supported by dose distribution information using the DIR algorithm implemented in the Velocity program to confirm our model (Table 3).…”

Creating patient‐specific digital phantoms with a longitudinal atlas for evaluating deformable CT‐CBCT registration in adaptive lung radiotherapy

“…The use of digital phantoms is recommended by AAPM Task Group 132 9 as part of the commissioning tests for DIR algorithms and has been widely investigated using a combination of biomechanical and thin plate spline algorithms, [21][22][23] finite element modeling, 24,25 or geometric transformations. 9 While each approach has its own advantage and dedicated application, our solution combines the following three advantages.…”

“…14,15 Insufficient ground truth information is the major obstacle to a comprehensive evaluation of DIR.Creation of useful digital or physical anthropomorphic phantoms has been intensively investigated to mitigate this problem. [16][17][18][19][20][21][22][23][24][25] However, the algorithms driving the deformation are often limited to a specific application or a particular clinical site, and the created phantoms are often not centered around an individual patient of interest to properly address the patient-specific characteristics. Moreover, current approaches do not consider the anatomical changes and image quality encountered in the longitudinal imaging data relevant for adaptive radiotherapy.…”

Review and recommendations on deformable image registration uncertainties for radiotherapy applications

“…The choice of the DIR algorithm and parameter settings influences the DVF obtained when registering the same image pair. Several studies investigate the performance of different DIR algorithms, for example in HN (Hardcastle et al 2012 , Močnik et al 2018 , Qin et al 2018 , Lee et al 2020 , Kubli et al 2021 ), lung (Kadoya et al 2014 , Scaggion et al 2020a ), liver (Zhang et al 2012 , Sen et al 2020 ) or pelvis (Hammers et al 2020 ). Some commercial DIR algorithms offer the possibility of parameter adjustments, such as registration metrics, guiding structures, regularisation levels, regularisation weights, or contrast level sensitivity, which causes uncertainty of the algorithm to vary (Ziegler et al 2019 ).…”