Accurate segmentation of head and neck radiotherapy CT scans with 3D CNNs: consistency is key

Abstract

Objective: Automatic segmentation of organs-at-risk in radiotherapy planning CT scans using convolutional neural networks (CNNs) is an active research area. Very large datasets are usually required to train such CNN models. In radiotherapy, large, high-quality datasets are scarce and combining data from several sources can reduce the consistency of training segmentations. It is therefore important to understand the impact of training data quality on the performance of auto-segmentation models for radiotherapy.
Approach: In this study, we took an existing 3D CNN architecture for head and neck CT auto-segmentation and compare the performance of models trained with a small, well-curated dataset (n=34) and then a far larger dataset (n=185) containing less consistent training segmentations. We performed 5-fold cross-validations in each dataset and tested segmentation performance using the 95th percentile Hausdorff distance and mean distance-to-agreement metrics. Finally, we validated the generalisability of our models with an external cohort of patient data (n=12) with five expert annotators.
Main results: The models trained with a large dataset were greatly outperformed by models (of identical architecture) trained with a smaller, but higher consistency set of training samples. Our models trained with a small dataset produce segmentations of similar accuracy as expert human observers and generalised well to new data, performing within inter-observer variation.
Significance: We empirically demonstrate the importance of highly consistent training samples when training a 3D auto-segmentation model for use in radiotherapy. Crucially, it is the consistency of the training segmentations which had a greater impact on model performance rather than the size of the dataset used.

Evolving Horizons in Radiation Therapy Auto-Contouring: Distilling Insights, Embracing Data-Centric Frameworks, and Moving Beyond Geometric Quantification

“…A tangible manifestation of the "garbage in, garbage out" principle within HNC contouring is exemplified in a study by Henderson et al 9 Their findings revealed that models trained on a small set of consistent contours (ie, strictly following guidelines) aligned more closely with the reference standard test data than those trained on a vast array of inconsistent contours ( Fig. 1 ).…”

Deep Learning and Registration-Based Mapping for Analyzing the Distribution of Nodal Metastases in Head and Neck Cancer Cohorts: Informing Optimal Radiotherapy Target Volume Design

“…The planning CT datasets available for the present analysis had a slice thickness of 3 mm, which is in the range radiotherapy head and neck image analysis methods are currently developed and evaluated on [30,42]. A lower slice thickness will increase the resolution in the z-dimension but also the computational time for the mapping analysis.…”

“…Given that FDG PET has exhibited significantly higher specificity and numerically superior sensitivity compared to contrast-enhanced CT imaging in identifying metastatic lymph nodes in the head and neck [ 41 ], the application of the proposed image analysis techniques to extensive cohorts with FDG PET imaging holds particular promise. The planning CT datasets available for the present analysis had a slice thickness of 3 mm, which is in the range radiotherapy head and neck image analysis methods are currently developed and evaluated on [ 30 , 42 ]. A lower slice thickness will increase the resolution in the z-dimension but also the computational time for the mapping analysis.…”

Artificial intelligence to unlock real‐world evidence in clinical oncology: A primer on recent advances

“…These algorithms, primarily based on CNNs, have shown remarkable accuracy and efficiency in delineating OARs and tumor targets and have rapidly entered routine clinical practice 87 . A noteworthy development is the integration of 3D CNNs, which better capture the spatial relationships in volumetric data, leading to improved contour accuracy compared to traditional approaches 88 . Generative adversarial networks (GANs) have been employed to augment training datasets and to perform data normalization, enhancing model robustness across different imaging modalities and protocols 89 .…”

“… 87 A noteworthy development is the integration of 3D CNNs, which better capture the spatial relationships in volumetric data, leading to improved contour accuracy compared to traditional approaches. 88 Generative adversarial networks (GANs) have been employed to augment training datasets and to perform data normalization, enhancing model robustness across different imaging modalities and protocols. 89 AI applications in radiation treatment planning have also advanced notably.…”