A direct morphometric comparison of five labeling protocols for multi-atlas driven automatic segmentation of the hippocampus in Alzheimer's disease

Abstract

Hippocampal volumetry derived from structural MRI is increasingly used to delineate regions of interest for functional measurements, assess efficacy in therapeutic trials of Alzheimer’s disease (AD) and has been endorsed by the new AD diagnostic guidelines as a radiological marker of disease progression. Unfortunately, morphological heterogeneity in AD can prevent accurate demarcation of the hippocampus. Recent developments in automated volumetry commonly use multitemplate fusion driven by expert manual labels, enabling highly accurate and reproducible segmentation in disease and healthy subjects. However, there are several protocols to define the hippocampus anatomically in vivo, and the method used to generate atlases may impact automatic accuracy and sensitivity – particularly in pathologically heterogeneous samples. Here we report a fully automated segmentation technique that provides a robust platform to directly evaluate both technical and biomarker performance in AD among anatomically unique labeling protocols. For the first time we test head-to-head the performance of five common hippocampal labeling protocols for multi-atlas based segmentation, using both the Sunnybrook Longitudinal Dementia Study and the entire Alzheimer’s Disease Neuroimaging Initiative 1 (ADNI-1) baseline and 24-month dataset. We based these atlas libraries on the protocols of (Haller et al., 1997; Killiany et al., 1993; Malykhin et al., 2007; Pantel et al., 2000; Pruessner et al., 2000), and a single operator performed all manual tracings to generate de facto “ground truth” labels. All methods distinguished between normal elders, mild cognitive impairment (MCI), and AD in the expected directions, and showed comparable correlations with measures of episodic memory performance. Only more inclusive protocols distinguished between stable MCI and MCI-to-AD converters, and had slightly better associations with episodic memory. Moreover, we demonstrate that protocols including more posterior anatomy and dorsal white matter compartments furnish the best voxel-overlap accuracies (Dice Similarity Coefficient = 0.87–0.89), compared to expert manual tracings, and achieve the smallest sample sizes required to power clinical trials in MCI and AD. The greatest distribution of errors was localized to the caudal hippocampus and alveus-fimbria compartment when these regions were excluded. The definition of the medial body did not significantly alter accuracy among more comprehensive protocols. Voxel-overlap accuracies between automatic and manual labels were lower for the more pathologically heterogeneous Sunnybrook study in comparison to the ADNI-1 sample. Finally, accuracy among protocols appears to significantly differ the most in AD subjects compared to MCI and normal elders. Together, these results suggest that selection of a candidate protocol for fully automatic multi-template based segmentation in AD can influence both segmentation accuracy when compared to expert manual labels and performance as a biomarker in MCI and AD.

Amygdalar and hippocampal volume: A comparison between manual segmentation, Freesurfer and VBM

“…It is not only difficult to estimate the "true" amygdala or hippocampal volumes, but it is additionally difficult to find agreement which measures are best suited for such an analysis. In addition to the measures we employed, several measures of quality such as Hausdorff distances of contours and surfaces, surface coverage or Dice and Jaccard coefficients have been discussed (cf., Bankman, 2008;Pham et al, 2000;Nestor et al, 2012). We did not use the Dice coefficient, as we were interested in a widely used univariate measure such as brain volume.…”

A comparative study of automatic image segmentation algorithms for target tracking in MR‐IGRT

“…At the same time, the TRE value that captures the overall position of the target ROI is important for the control of radiotherapy delivery. Manual contours have been serving as ground truth as an established practice — for example in segmenting brain tumor, (33) hippocampus region, (34) esophageal and gastroesophageal cancer, (35) among others. Although there is variability in physician's contour of regions of interest, it is a known limitation of studies evaluating efficacy of segmentation algorithms in medical imaging (36) .…”

Standardization of MRI and Amyloid Imaging

“…Even within the narrower field of AD research, 12 different widely used segmentation protocols could be identified, leading to up to 2.5-fold volume differences in reported group means for hippocampus volume [112]. This variability in hippocampus definition equally affects automated segmentation methods, which depend on accurate hippocampus labels in template space [113]. Beyond hippocampus volumetry, a wide variety of automated methods for the quantitative assessment of regional volumetric changes in AD has been proposed, mainly differing in the selection of assessed brain regions [60,[114][115][116][117][118].…”

“…The standardization of manual hippocampus delineation in structural MRI scans will also benefit harmonization of automated segmentation algorithms because of their dependence on a-priori models of hippocampal shape [113]. As an extension of the harmonization project, a larger dataset of benchmark labels covering a wide physiological variability across healthy and diseased brains is currently being produced by qualified human tracers to produce a comprehensive set of gold standard labels suitable for proper training of automated segmentation algorithms.…”