Evaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge

Abstract

EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

Differences in brain morphology and working memory capacity across childhood

“…For comparison across participants, we created a study‐specific FA‐template based on all available images using Advanced Normalization Tools (ANTs) algorithms (Avants et al., 2014; Lawson, Duda, Avants, Wu, & Farah, 2013), which showed the highest accuracy in software comparisons (Klein et al., 2009; Murphy et al., 2011; Tustison et al., 2014). Individual images were transformed to template space using non‐linear registration with symmetric diffeomorphic normalization as implemented in ANTs (Avants, Epstein, Grossman, & Gee, 2008).…”

CT ventilation functional image-based IMRT treatment plans are comparable to SPECT ventilation functional image-based plans

“…For reconstruction, the CT images (GE) or projections (Siemens) were sorted into ten respiratory bins by the phase-based method using GE Advantage 4D or Siemens Biograph 40 software. The second step was deformable image registration (DIR) for spatial mapping of the peak-inhale 4D-CT image data set (moving) to the peak-exhale image data set (fixed) using a volumetric elastic DIR method, which was found to have sub-voxel accuracy in the previous studies [23][24][25]. The same level of accuracy was assumed in this study.…”

Benchmarking of five commercial deformable image registration algorithms for head and neck patients

“…In an effort to provide consistent datasets for algorithm validation, several researchers have made their ground‐truth models publicly available. Examples include the extended cardiac–torso (XCAT) phantom, (3) the point‐validated pixel‐based breathing thorax model (POPI model), (4) the DIR‐Lab Thoracic 4D CT model, 5 , 6 and those provided as part of the EMPIRE 10 challenge (7) . In an effort to improve the correlation of computer‐based phantoms with the actual anatomical changes seen over a course of radiation therapy, the current authors previously developed synthetic datasets derived from clinical images of real head and neck patients (8) .…”