Multi-planar whole heart segmentation of 3D CT images using 2D spatial propagation CNN

Abstract

Whole heart segmentation from cardiac CT scans is a prerequisite for many clinical applications, but manual delineation is a tedious task and subject to both intra-and inter-observer variation. Automating the segmentation process has thus become an increasingly popular task in the field of image analysis, and is generally solved by either using 3D methods, considering the image volume as a whole, or 2D methods, segmenting each slice independently. In the field of deep learning, there are significant limitations regarding 3D networks, including the need for more training examples and GPU memory. The need for GPU memory is usually solved by down sampling the input images, thus losing important information, which is not a necessary sacrifice when employing 2D networks. It would therefore be relevant to exploit the benefits of 2D networks in a configuration, where spatial information across slices is kept, as when employing 3D networks. The proposed method performs multiclass segmentation of cardiac CT scans utilizing 2D convolutional neural networks with a multi-planar approach. Furthermore, spatial propagation is included in the network structure, to ensure spatial consistency through each image volume. The approach keeps the computational assets of 2D methods while addressing 3D issues regarding spatial context. The pipeline is structured in a two-step approach, in which the first step detects the location of the heart and crops a region of interest, and the second step performs multi-class segmentation of the heart structures. The pipeline demonstrated promising results on the MICCAI 2017 Multi-Modality Whole Heart Segmentation challenge data.

Synthetic Data Generation for Automatic Segmentation of X-ray Computed Tomography Reconstructions of Complex Microstructures

“…For 3D DCNNs, the limitation is the vast number of required training data. Some researchers have overcome similar problems by averaging resulting output segmentation probabilities from multiple plane views with 2D networks [ 15 , 16 ]. Nonetheless, manually annotated labels are still required for training, with the number of slices needed depending on the complexity of the problem to be tackled.…”

Whole Heart Segmentation Using 3D FM-Pre-ResNet Encoder–Decoder Based Architecture with Variational Autoencoder Regularization

“…Finally, they apply a hierarchical shape prior algorithm [17] on the probability map to estimate the shape of each heart structure. Sundgaard et al [18] use 2D CNNs with a multi-planar method to investigate the power of retaining spatial information across slices, as is the case of 3D networks. Mortazi et al [19] present a multi-planar CNN method using an encoder-decoder architecture.…”