Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?

Bernard, Olivier; Lalande, Alain; Zotti, Clément; Cervenansky, Frederick; Yang, Xin; Heng, Pheng‐Ann; Cetin, Irem; Lekadir, Karim; Cámara, Óscar; Ballester, Miguel Ángel González; Sanromà, Gerard; Napel, Sandy; Petersen, Steffen E.; Tziritas, Georgios; Grinias, Elias; Khened, Mahendra; Kollerathu, Varghese Alex; Krishnamurthi, Ganapathy; Rohé, Marc-Michel; Pennec, Xavier; Sermesant, Maxime; Isensee, Fabian; Jäger, Paul F.; Maier‐Hein, Klaus H.; Full, Peter M.; Wolf, Ivo; Engelhardt, Sandy; Baumgartner, Christian F.; Koch, Lisa M.; Wolterink, Jelmer M.; Išgum, Ivana; Jang, Yeonggul; Hong, Yoonmi; Patravali, Jay; Jain, Shubham; Humbert, Olivier; Jodoin, Pierre-Marc

doi:10.1109/tmi.2018.2837502

Cited by 1,282 publications

(929 citation statements)

References 51 publications

(78 reference statements)

Supporting

Mentioning

834

Contrasting

Unclassified

Order By: Relevance

“…From Fig. 4(a), we can observe consistent improvement in segmentation performance at the problematic apical and basal slices; however, due to the small size of these regions, the improvement does not have a large effect on the overall performance, though it is of significance when constructing patient specific models of the heart for simulation purposes . We postulate that the additional constraint imposed by a very high negative distance assigned to empty apical/basal slices prevents the network from oversegmenting these regions, hence, improving the regional dice overlap and effectively reducing the overall Hausdorff distance.…”

Section: Resultsmentioning

confidence: 79%

“…This dataset † is composed of short-axis cardiac cine-MR images acquired for 150 patients divided into 5 evenly distributed subgroups: normal, myocardial infarction, dilated cardiomyopathy, hypertropic cardiomyopathy, and abnormal right ventricle, available as a part of the STACOM 2017 ACDC challenge. 39 The acquisitions were obtained over a 6-yr period using two MRI scanners of different magnetic strengths (1.5 and 3.0 T). The images were acquired using the SSFP sequence with the following settings: thickness 5 mm (sometimes 8 mm), interslice gap 5 mm, spatial resolution 1.37-1.68 mm 2 =pixel, 28-40 frames per cardiac cycle.…”

Section: D2 Automated Cardiac Diagnosis Challenge (Acdc)mentioning

confidence: 99%

See 1 more Smart Citation

A distance map regularized CNN for cardiac cine MR image segmentation

2019

View full text Add to dashboard Cite

Purpose Cardiac image segmentation is a critical process for generating personalized models of the heart and for quantifying cardiac performance parameters. Fully automatic segmentation of the left ventricle (LV), the right ventricle (RV), and the myocardium from cardiac cine MR images is challenging due to variability of the normal and abnormal anatomy, as well as the imaging protocols. This study proposes a multi‐task learning (MTL)‐based regularization of a convolutional neural network (CNN) to obtain accurate segmenation of the cardiac structures from cine MR images. Methods We train a CNN network to perform the main task of semantic segmentation, along with the simultaneous, auxiliary task of pixel‐wise distance map regression. The network also predicts uncertainties associated with both tasks, such that their losses are weighted by the inverse of their corresponding uncertainties. As a result, during training, the task featuring a higher uncertainty is weighted less and vice versa. The proposed distance map regularizer is a decoder network added to the bottleneck layer of an existing CNN architecture, facilitating the network to learn robust global features. The regularizer block is removed after training, so that the original number of network parameters does not change. The trained network outputs per‐pixel segmentation when a new patient cine MR image is provided as an input. Results We show that the proposed regularization method improves both binary and multi‐class segmentation performance over the corresponding state‐of‐the‐art CNN architectures. The evaluation was conducted on two publicly available cardiac cine MRI datasets, yielding average Dice coefficients of 0.84 ± 0.03 and 0.91 ± 0.04. We also demonstrate improved generalization performance of the distance map regularized network on cross‐dataset segmentation, showing as much as 42% improvement in myocardium Dice coefficient from 0.56 ± 0.28 to 0.80 ± 0.14. Conclusions We have presented a method for accurate segmentation of cardiac structures from cine MR images. Our experiments verify that the proposed method exceeds the segmentation performance of three existing state‐of‐the‐art methods. Furthermore, several cardiac indices that often serve as diagnostic biomarkers, specifically blood pool volume, myocardial mass, and ejection fraction, computed using our method are better correlated with the indices computed from the reference, ground truth segmentation. Hence, the proposed method has the potential to become a non‐invasive screening and diagnostic tool for the clinical assessment of various cardiac conditions, as well as a reliable aid for generating patient specific models of the cardiac anatomy for therapy planning, simulation, and guidance.

show abstract

Section: Resultsmentioning

confidence: 79%

Section: D2 Automated Cardiac Diagnosis Challenge (Acdc)mentioning

confidence: 99%

A distance map regularized CNN for cardiac cine MR image segmentation

2019

View full text Add to dashboard Cite

show abstract

“…In this work, we used a dilated CNN (DCNN) for segmentation. This architecture has previously shown excellent performance on medical image analysis tasks . The DCNN was trained to segment 2D slices in the transversal, sagittal, and coronal images.…”

Section: Discussionmentioning

confidence: 99%

Knowledge‐based and deep learning‐based automated chest wall segmentation in magnetic resonance images of extremely dense breasts

et al. 2019

Self Cite

View full text Add to dashboard Cite

Purpose Segmentation of the chest wall, is an important component of methods for automated analysis of breast magnetic resonance imaging (MRI). Methods reported to date show promising results but have difficulties delineating the muscle border correctly in breasts with a large proportion of fibroglandular tissue (i.e., dense breasts). Knowledge‐based methods (KBMs) as well as methods based on deep learning have been proposed, but a systematic comparison of these approaches within one cohort of images is currently lacking. Therefore, we developed a KBM and a deep learning method for segmentation of the chest wall in MRI of dense breasts and compared their performances. Methods Two automated methods were developed, an optimized KBM incorporating heuristics aimed at shape, location, and gradient features, and a deep learning‐based method (DLM) using a dilated convolution neural network. A data set of 115 T1‐weighted MR images was randomly selected from MR images of women with extremely dense breasts (ACR BI‐RADS category 4) participating in a screening trial of women (mean age 56.6 yr, range 49.5–75.2 yr) with dense breasts. Manual segmentations of the chest wall, acquired under supervision of an experienced breast radiologist, were available for all data sets. Both methods were optimized using the same randomly selected 36 MRI data sets from a total of 115 data sets. Each MR data set consisted of 179 transversal images with voxel size 0.64 mm3 × 0.64 mm3 × 1.00 mm3. In the remaining 79 data sets, the results of both segmentation methods were qualitatively evaluated. A radiologist reviewed the segmentation results of both methods in all transversal images (n = 14 141) and determined whether the result would impact the ability to accurately determine the volume of fibroglandular and fatty tissue and whether segmentations masked breast regions that might harbor lesions. When no relevant deviation was detected, the result was considered successful. In addition, all segmentations were quantitatively assessed using the Dice similarity coefficient (DSC) and Hausdorff distance (HD), 95th percentile of the Hausdorff distance (HD95), false positive fraction (FPF), and false negative fraction (FNF) metrics. Results According to the radiologist's evaluation, the DLM had a significantly higher success rate than the KBM (81.6% vs 78.4%, P < 0.01). The success rate was further improved to 92.1% by combining both methods. Similarly, the DLM had significantly lower values for FNF (0.003 ± 0.003 vs 0.009 ± 0.011, P < 0.01) and HD95 (2.58 ± 1.78 mm vs 3.37 ± 2.11, P < 0.01). However, the KBM resulted in a significantly lower FPF than the DLM (0.018 ± 0.009 vs 0.030 ± 0.009, P < 0.01).There was no significant difference between the KBM and DLM in terms of DSC (0.982 ± 0.006 vs 0.984 ± 0.008, P = 0.08) or HD (24.14 ± 20.69 mm vs 12.81 ± 27.28 mm, P = 0.05). Conclusion Both optimized knowledge‐based and DLM showed good results to segment the pectoral muscle in women with dense breasts. Qualitatively assessed, the DLM was the most robust...

show abstract

“…Finally, we introduce a batch‐wise weighted Dice loss function to improve the training process with unbalanced data. The proposed method was evaluated and analyzed on the MICCAI 2017 ACDC dataset and further validated on the Sunnybrook dataset to investigate its applicability to other datasets.…”

Section: Introductionmentioning

confidence: 99%

“…Cardiac function analysis plays an important role in clinical cardiology for disease diagnosis, risk evaluation, patient management, and therapy decision. 1 Cine cardiac magnetic resonance imaging (MRI) is widely used for cardiac function analysis through the estimation of clinical parameters such as ejection fraction, stroke volume, ventricular volume, and myocardial thickness. This requires accurate segmentation of the heart structures such as the left ventricle (LV), right ventricle (RV) cavities, and the myocardium (MYO) from the corresponding cardiac MR images.…”

Section: Introductionmentioning

confidence: 99%

An iterative multi‐path fully convolutional neural network for automatic cardiac segmentation in cine MR images

Wang³

et al. 2019

Medical Physics

View full text Add to dashboard Cite

Purpose Segmentation of the left ventricle (LV), right ventricle (RV) cavities and the myocardium (MYO) from cine cardiac magnetic resonance (MR) images is an important step for diagnosis and monitoring cardiac diseases. Spatial context information may be highly beneficial for segmentation performance improvement. To this end, this paper proposes an iterative multi‐path fully convolutional network (IMFCN) to effectively leverage spatial context for automatic cardiac segmentation in cine MR images. Methods To effectively leverage spatial context information, the proposed IMFCN explicitly models the interslice spatial correlations using a multi‐path late fusion strategy. First, the contextual inputs including both the adjacent slices and the already predicted mask of the above adjacent slice are processed by independent feature‐extraction paths. Then, an atrous spatial pyramid pooling (ASPP) module is employed at the feature fusion process to combine the extracted high‐level contextual features in a more effective way. Finally, deep supervision (DS) and batch‐wise class re‐weighting mechanism are utilized to enhance the training of the proposed network. Results The proposed IMFCN was evaluated and analyzed on the MICCAI 2017 automatic cardiac diagnosis challenge (ACDC) dataset. On the held‐out training dataset reserved for testing, our method effectively improved its counterparts that without spatial context and that with spatial context but using an early fusion strategy. On the 50 subjects test dataset, our method achieved Dice similarity coefficient of 0.935, 0.920, and 0.905, and Hausdorff distance of 7.66, 12.10, and 8.80 mm for LV, RV, and MYO, respectively, which are comparable or even better than the state‐of‐the‐art methods of ACDC Challenge. In addition, to explore the applicability to other datasets, the proposed IMFCN was retrained on the Sunnybrook dataset for LV segmentation and also produced comparable performance to the state‐of‐the‐art methods. Conclusions We have presented an automatic end‐to‐end fully convolutional architecture for accurate cardiac segmentation. The proposed method provides an effective way to leverage spatial context in a two‐dimensional manner and results in precise and consistent segmentation results.

show abstract

Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?

Cited by 1,282 publications

References 51 publications

A distance map regularized CNN for cardiac cine MR image segmentation

A distance map regularized CNN for cardiac cine MR image segmentation

Knowledge‐based and deep learning‐based automated chest wall segmentation in magnetic resonance images of extremely dense breasts

An iterative multi‐path fully convolutional neural network for automatic cardiac segmentation in cine MR images

Contact Info

Product

Resources

About