Development and validation of a quality control procedure for automatic segmentation of hippocampal subfields

Abstract

Automatic segmentation methods for in vivo magnetic resonance imaging are increasing in popularity because of their high efficiency and reproducibility. However, automatic methods can be perfectly reliable and consistently wrong, and the validity of automatic segmentation methods cannot be taken for granted. Quality control (QC) by trained and reliable human raters is necessary to ensure the validity of automatic measurements. Yet QC practices for applied neuroimaging research are underdeveloped. We report a detailed QC and correction procedure to accompany our validated atlas for hippocampal subfield segmentation. We document a two‐step QC procedure for identifying segmentation errors, along with a taxonomy of errors and an error severity rating scale. This detailed procedure has high between‐rater reliability for error identification and manual correction. The latter introduces at maximum 3% error variance in volume measurement. All procedures were cross‐validated on an independent sample collected at a second site with different imaging parameters. The analysis of error frequency revealed no evidence of bias. An independent rater with a third sample replicated procedures with high within‐rater reliability for error identification and correction. We provide recommendations for implementing the described method along with hypothesis testing strategies. In sum, we present a detailed QC procedure that is optimized for efficiency while prioritizing measurement validity and suits any automatic atlas.

A (Sub)field Guide to Quality Control in Hippocampal Subfield Segmentation on High-resolution T2-weighted MRI

“…Example QC approach using a 4-point error severity scale from a published and validated protocol (Canada et al, 2023). In this example protocol, errors could be 0- not present (not pictured here), 1- minor (<10% of label affected), 2- moderate (10–25% of label affected), or 3- major (>25% label affected) and were categorized by type.…”

“…There are several resources available to provide some options for standardized protocols: 1) MAGeT-related QC (https://github.com/CoBrALab/documentation/wiki/MAGeT-Brain-Quality-Control-(QC)-Guide), 2) HippUnfold’s automated QC (DSC overlap with a deformable registration), and 3) MRIQC (Esteban et al, 2017). Additionally, the examples of manual evaluations from Canada et al (2023) and Wisse (https://www.youtube.com/watch?v=XHXu-AGR6pE) demonstrate that investigators can use different criteria but similarly implement the recommended best practice to identify severe segmentation errors. Using existing QC protocols as a resource provides operational definitions and criterion that can be applied or modified for an investigator’s particular data set.…”

“…Selective manual correction of only some errors, such as removing an obvious mislabel (e.g., cerebrospinal fluid) or adding labels to voxels that should have been labeled originally, is a viable approach to addressing severe segmentation errors while preserving benefits of automated segmentation efficiency, especially when working with large data sets. The severity ratings of segmentation errors made in prior QC steps are used to determine where manual intervention is applied (e.g., Canada et al, 2023). In this practice, manual corrections are made according to the segmentation protocol that was used to generate the segmentations which allows all images to be segmented using the same protocol regardless of whether it was corrected.…”

Specific amygdala and hippocampal subfield volumes in social anxiety disorder and their relation to clinical characteristics – an international mega-analysis

“…Given the roles of the amygdala and hippocampus in fear and anxiety 20,21 , and developments in amygdala and hippocampal segmentation techniques (Saygin et al, 2017), there have been increasing reports of subfield volume alterations in anxiety-related disorders 22 . Reviews of the hippocampal subfields literature suggest that individuals with MDD have smaller volumes of the CA3/4 and larger volume of the hippocampus–amygdala transition area (HATA) compared to HCs 23 , while individuals with PTSD have smaller volumes of the CA1/3 and dentate gyrus (DG) compared to HCs (Ben-Zion et al, 2023).…”