Deep learning for automatic head and neck lymph node level delineation provides expert-level accuracy

Abstract

BackgroundDeep learning-based head and neck lymph node level (HN_LNL) autodelineation is of high relevance to radiotherapy research and clinical treatment planning but still underinvestigated in academic literature. In particular, there is no publicly available open-source solution for large-scale autosegmentation of HN_LNL in the research setting.MethodsAn expert-delineated cohort of 35 planning CTs was used for training of an nnU-net 3D-fullres/2D-ensemble model for autosegmentation of 20 different HN_LNL. A second cohort acquired at the same institution later in time served as the test set (n = 20). In a completely blinded evaluation, 3 clinical experts rated the quality of deep learning autosegmentations in a head-to-head comparison with expert-created contours. For a subgroup of 10 cases, intraobserver variability was compared to the average deep learning autosegmentation accuracy on the original and recontoured set of expert segmentations. A postprocessing step to adjust craniocaudal boundaries of level autosegmentations to the CT slice plane was introduced and the effect of autocontour consistency with CT slice plane orientation on geometric accuracy and expert rating was investigated.ResultsBlinded expert ratings for deep learning segmentations and expert-created contours were not significantly different. Deep learning segmentations with slice plane adjustment were rated numerically higher (mean, 81.0 vs. 79.6, p = 0.185) and deep learning segmentations without slice plane adjustment were rated numerically lower (77.2 vs. 79.6, p = 0.167) than manually drawn contours. In a head-to-head comparison, deep learning segmentations with CT slice plane adjustment were rated significantly better than deep learning contours without slice plane adjustment (81.0 vs. 77.2, p = 0.004). Geometric accuracy of deep learning segmentations was not different from intraobserver variability (mean Dice per level, 0.76 vs. 0.77, p = 0.307). Clinical significance of contour consistency with CT slice plane orientation was not represented by geometric accuracy metrics (volumetric Dice, 0.78 vs. 0.78, p = 0.703).ConclusionsWe show that a nnU-net 3D-fullres/2D-ensemble model can be used for highly accurate autodelineation of HN_LNL using only a limited training dataset that is ideally suited for large-scale standardized autodelineation of HN_LNL in the research setting. Geometric accuracy metrics are only an imperfect surrogate for blinded expert rating.

Review of Deep Learning Based Autosegmentation for Clinical Target Volume: Current Status and Future Directions

“…Generally, few edits of the DLAS model contours were required. 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 Wong et al found the commercial DLAS model that was used had worse performance (DSC, 0.72) compared with manual contouring, although the model led to fast contouring of CTV volumes. More data are needed to compare the performance of commercial and in-house DLAS models.…”

“… 53 Weissman et al reported improved DLAS performance when the model was adjusted to the CT slice plane compared with when the model was not adjusted to the CT slice plane. 55 van der Veen et al reported best DLAS performance for LN levels Ib, II-IVa, VIa, VIb, VIIa, and VIIb (DSC, 0.85), and Kihara et al reported their DLAS model incorrectly segmented 1b lymph node levels for tonsillar and base of tongue cancer. 52 , 57 van der Veen et al also measured time to DLAS of all lymph node levels to be 86 seconds with the time needed to correct autosegmented contours of lymph nodes (35 minutes) to be less than time to correct manual contours (52 minutes).…”

Evolving Horizons in Radiation Therapy Auto-Contouring: Distilling Insights, Embracing Data-Centric Frameworks, and Moving Beyond Geometric Quantification

“…Interestingly, the study illustrated diminishing returns as the training set size increased, with performance plateauing or even declining in some cases, a trend potentially attributed to the negative effects of inconsistent annotation in the authors’ data set. Similarly, Yu et al 13 and Weissmann et al 14 demonstrated that small, well-curated data sets can be used to train publicly available models to achieve clinically acceptable results.

Figure 3 Relatively small training sample sizes are needed to reach high geometric performance for deep learning auto-contouring models.

…”

Clinical acceptability of automatically generated lymph node levels and structures of deglutition and mastication for head and neck radiation therapy

“…Automatic CT-based segmentation of these lymph node levels (e.g. I-V) is achievable and has been demonstrated by numerous works [10] , [21] , [22] , [23] , [24] , [25] , [26] , [27] . Our clinic previously integrated a deep-learning based approach to contour elective lymph node levels in CT scans [10] .…”