Multi-Planar Deep Segmentation Networks for Cardiac Substructures from MRI and CT

Abstract

Non-invasive detection of cardiovascular disorders from radiology scans requires quantitative image analysis of the heart and its substructures. There are well-established measurements that radiologists use for diseases assessment such as ejection fraction, volume of four chambers, and myocardium mass. These measurements are derived as outcomes of precise segmentation of the heart and its substructures. The aim of this paper is to provide such measurements through an accurate image segmentation algorithm that automatically delineates seven substructures of the heart from MRI and/or CT scans. Our proposed method is based on multi-planar deep convolutional neural networks (CNN) with an adaptive fusion strategy where we automatically utilize complementary information from different planes of the 3D scans for improved delineations. For CT and MRI, we have separately designed three CNNs (the same architectural configuration) for three planes, and have trained the networks from scratch for voxel-wise labeling for the following cardiac structures: myocardium of left ventricle (Myo), left atrium (LA), left ventricle (LV), right atrium (RA), right ventricle (RV), ascending aorta (Ao), and main pulmonary artery (PA). We have evaluated the proposed method with 4-fold-cross-validation on the multi-modality whole heart segmentation challenge (MM-WHS 2017) dataset. The precision and dice index of 0.93 and 0.90, and 0.87 and 0.85 were achieved for CT and MR images, respectively. While a CT volume was segmented about 50 seconds, an MRI scan was segmented around 17 seconds with the GPUs/CUDA implementation.

Cardiac substructure segmentation with deep learning for improved cardiac sparing

“…17 Nevertheless, our DL pipeline performed well for coronary artery contours on non-contrast CTs (DSC~0.50, MDA < 2.0 mm), particularly as compared to recent atlas results where coronary artery (LADA, RCA, and LMCA) DSCs ranged from 0.09 to 0.27 [13][14][15] and had MDAs > 4 mm. 14 Coronary artery segmentations may be improved through the use of high resolution (0.78 9 0.78 9 1.6 mm 3 ) CTCA 21 that use contrast and yield DSCs~60%. 52 Additionally, implementing a Dice loss function weighted on the inverse of the class size may improve the results for smaller substructures such as the 53 the generalized Dice loss (GDL) function has been shown to improve hyperparameter robustness for unbalanced tasks (i.e., when each class is not represented equally in the dataset), and improve overall segmentation accuracy for small structures.…”

“…This value can be compared to Mortazi et al who segmented seven cardiac substructures in~50 s on high resolution CTCA and 17 s on MRI. 21 Moreover, our previous MA method required~10 min to generate substructure contours per patient without post-processing. 13 Although the in-plane resolution was 0.7 9 0.7 mm 2 , our study may have been limited by the 8 mm slice thickness of the MRI.…”

Deep Learning for Cardiac Image Segmentation: A Review

“…Multi-view CNNs: Another line of research utilizes the volumetric information of the heart by training multi-planar CNNs (axial, sagittal, and coronal views) in a 2D fashion. Examples include Wang and Smedby (2017) and Mortazi et al (2017a) where three independent orthogonal CNNs were trained to segment different views. Specifically, Wang and Smedby (2017) additionally incorporated shape context in the framework for the segmentation refinement, while Mortazi et al (2017a) adopted an adaptive fusion strategy to combine multiple outputs utilising complementary information from different planes.…”

Deep Learning in Cardiology

“…The model was compared with u-net achieving superior results, especially in the MRI dataset. Mortazi et al [153] trained a multi-planar CNN with an adaptive fusion strategy for segmenting seven substructures of the heart. They designed three CNNs, one for each plane, with the same architectural configuration and trained them for voxel-wise labeling.…”