Interleaved 3D‐CNNs for joint segmentation of small‐volume structures in head and neck CT images

Abstract

Purpose Accurate 3D image segmentation is a crucial step in radiation therapy planning of head and neck tumors. These segmentation results are currently obtained by manual outlining of tissues, which is a tedious and time-consuming procedure. Automatic segmentation provides an alternative solution, which, however, is often difficult for small tissues (i.e., chiasm and optic nerves in head and neck CT images) because of their small volumes and highly diverse appearance/shape information. In this work, we propose to interleave multiple 3D Convolutional Neural Networks (3D-CNNs) to attain automatic segmentation of small tissues in head and neck CT images. Method A 3D-CNN was designed to segment each structure of interest. To make full use of the image appearance information, multi-scale patches are extracted to describe the center voxel under consideration and then input to the CNN architecture. Next, as neighboring tissues are often highly related in the physiological and anatomical perspectives, we interleave the CNNs designated for the individual tissues. In this way, the tentative segmentation result of a specific tissue can contribute to refine the segmentations of other neighboring tissues. Finally, as more CNNs are interleaved and cascaded, a complex network of CNNs can be derived, such that all tissues can be jointly segmented and iteratively refined. Result Our method was validated on a set of 48 CT images, obtained from the Medical Image Computing and Computer Assisted Intervention (MICCAI) Challenge 2015. The Dice coefficient (DC) and the 95% Hausdorff Distance (95HD) are computed to measure the accuracy of the segmentation results. The proposed method achieves higher segmentation accuracy (with the average DC: 0.58±0.17 for optic chiasm, and 0.71±0.08 for optic nerve; 95HD: 2.81±1.56 mm for optic chiasm, and 2.23±0.90 mm for optic nerve) than the MICCAI challenge winner (with the average DC: 0.38 for optic chiasm, and 0.68 for optic nerve; 95HD: 3.48 for optic chiasm, and 2.48 for optic nerve). Conclusion An accurate and automatic segmentation method has been proposed for small tissues in head and neck CT images, which is important for the planning of radiotherapy.

AnatomyNet: Deep learning for fast and fully automated whole‐volume segmentation of head and neck anatomy

“…Compared to the interleaved ConvNets in Ref. on small‐volumed organs, such as chiasm, optic nerve left, and optic nerve right, AnatomyNet is better on two of three cases. The interleaved ConvNets achieved higher performance on chiasm, which is likely contributed by the fact that its prediction was operated on small region of interest (ROI), obtained first through atlas registration, while AnatomyNet operates directly on whole‐volume slices.…”

“…) (Table ). We should note that this metric imposes more challenges to AnatomyNet than other methods operating on local patches (such as the method by Ren et al …”

“…Recently, deep convolutional models have shown great success for biomedical image segmentation and have been introduced to the field of HaN anatomy segmentation . However, the existing HaN‐related deep learning‐based methods either use sliding windows working on patches that cannot capture global features or rely on atlas registration to obtain highly accurate small regions of interest in the preprocessing.…”

“…Recently, deep convolutional models have shown great success for biomedical image segmentation 25 and have been introduced to the field of HaN anatomy segmentation. [26][27][28][29] However, the existing HaN-related deep learning-based methods either use sliding windows working on patches that cannot capture global features or rely on atlas registration to obtain highly accurate small regions of interest in the preprocessing. What is more appealing are models that receive the whole-volume image as input without heavy-duty preprocessing and then directly output the segmentations of all interested anatomies.…”

AnatomyNet: Deep 3D Squeeze-and-excitation U-Nets for fast and fully automated whole-volume anatomical segmentation

“…Ibragimov and Xing proposed a simple convolutional neural network for atlas-free deep learning based OARs segmentation [26]. Interleaved multiple 3D-CNN was proposed for small-volumed structure segmentation in the region of interest (ROI) obtained by atlas registration [27]. Hänsch et al conducted a comparison for different deep learning approaches for single anatomy, parotid gland, segmentation [28].…”