2D-3D Fully Convolutional Neural Networks for Cardiac MR Segmentation

Patravali, Jay; Jain, Shubham; Chilamkurthy, Sasank

doi:10.1007/978-3-319-75541-0_14

Cited by 79 publications

(66 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This contextual information can include shape priors learned from labels or multiview images (Zotti et al, 2017(Zotti et al, , 2019Chen et al, 2019b). Others extract spatial information from adjacent slices to assist the segmentation, using recurrent units (RNNs) or multi-slice networks (2.5D networks) (Poudel et al, 2016;Patravali et al, 2017;Du et al, 2019;Zheng et al, 2018).…”

Section: Ventricle Segmentationmentioning

confidence: 99%

“…LGE MR imaging enables the Table 2. Segmentation accuracy of state-of-the-art segmentation methods verified on the cardiac bi-ventricular segmentation challenge (ACDC) dataset All the methods were evaluated on the same test set (50 subjects 2D GridNet-MD with registered shape prior 0.938 0.894 0.910 Khened et al (2019) 2D Dense U-net with inception module 0.941 0.894 0.907 Baumgartner et al (2017) 2D U-net with cross entropy loss 0.937 0.897 0.908 Zotti et al (2017) 2D GridNet with registered shape prior 0.931 0.890 0.912 Jang et al (2017) 2D M-Net with weighted cross entropy loss 0.940 0.885 0.907 Painchaud et al (2019) FCN followed by an AE for shape correction 0.936 0.889 0.909 Wolterink et al (2017c) Multi-input 2D dilated FCN, segmenting paired ED and ES frames simultaneously 0.940 0.885 0.900 Patravali et al (2017) 2D U-net with a Dice loss 0.920 0.890 0.865 Rohé et al (2017) Multi-atlas based method combined with 3D CNN for registration 0.929 0.868 0.881 Tziritas and Grinias (2017) Level-set +markov random field (MRF); Non-deep learning method 0.907 0.798 0.803 Yang et al (2017c) 3D FCN with deep supervision 0.820 N/A 0.780…”

Section: Scar Segmentationmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for Cardiac Image Segmentation: A Review

Chen

Qiu

et al. 2020

Front. Cardiovasc. Med.

602

383

View full text Add to dashboard Cite

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research.

show abstract

Section: Ventricle Segmentationmentioning

confidence: 99%

Section: Scar Segmentationmentioning

confidence: 99%

Deep Learning for Cardiac Image Segmentation: A Review

Chen

Qiu

et al. 2020

Front. Cardiovasc. Med.

602

383

View full text Add to dashboard Cite

show abstract

“…However, in the context of cardiac image segmentation, due to the low through-plane resolution characteristic of cardiac MRI and the shortcomings of 3D methods such as reduction of training images and high risk of overfitting, 15,18 the segmentation performance of 3D methods may be limited to some extent. For instance, the previous work [19][20][21] evaluated on the automatic cardiac diagnosis challenge (ACDC) dataset indicated that the proposed 3D models did not meet the expectations in performance improvement over the corresponding 2D models. These motivate us to explore how to effectively leverage spatial context in 2D methods to combine the advantages of 2D and 3D methods.…”

Section: Introductionmentioning

confidence: 95%

“…Thus, the correlation between slices is weak except for adjacent slices. A direct way to use the contextual information of the adjacent slices is early fusion strategy, i.e., stacking them as input channels and output the prediction of the middle or target slice, as in the study of Patravali . This strategy treats the adjacent slices and the target slice equally and ignores the information provided by the already predicted segmentation.…”

Section: Introductionmentioning

confidence: 99%

An iterative multi‐path fully convolutional neural network for automatic cardiac segmentation in cine MR images

Wang³

et al. 2019

Medical Physics

View full text Add to dashboard Cite

Purpose Segmentation of the left ventricle (LV), right ventricle (RV) cavities and the myocardium (MYO) from cine cardiac magnetic resonance (MR) images is an important step for diagnosis and monitoring cardiac diseases. Spatial context information may be highly beneficial for segmentation performance improvement. To this end, this paper proposes an iterative multi‐path fully convolutional network (IMFCN) to effectively leverage spatial context for automatic cardiac segmentation in cine MR images. Methods To effectively leverage spatial context information, the proposed IMFCN explicitly models the interslice spatial correlations using a multi‐path late fusion strategy. First, the contextual inputs including both the adjacent slices and the already predicted mask of the above adjacent slice are processed by independent feature‐extraction paths. Then, an atrous spatial pyramid pooling (ASPP) module is employed at the feature fusion process to combine the extracted high‐level contextual features in a more effective way. Finally, deep supervision (DS) and batch‐wise class re‐weighting mechanism are utilized to enhance the training of the proposed network. Results The proposed IMFCN was evaluated and analyzed on the MICCAI 2017 automatic cardiac diagnosis challenge (ACDC) dataset. On the held‐out training dataset reserved for testing, our method effectively improved its counterparts that without spatial context and that with spatial context but using an early fusion strategy. On the 50 subjects test dataset, our method achieved Dice similarity coefficient of 0.935, 0.920, and 0.905, and Hausdorff distance of 7.66, 12.10, and 8.80 mm for LV, RV, and MYO, respectively, which are comparable or even better than the state‐of‐the‐art methods of ACDC Challenge. In addition, to explore the applicability to other datasets, the proposed IMFCN was retrained on the Sunnybrook dataset for LV segmentation and also produced comparable performance to the state‐of‐the‐art methods. Conclusions We have presented an automatic end‐to‐end fully convolutional architecture for accurate cardiac segmentation. The proposed method provides an effective way to leverage spatial context in a two‐dimensional manner and results in precise and consistent segmentation results.

show abstract

“…Existing popular fully-3D networks are smaller to reduce memory footprints, and may lack the capcity to learn challenging segmentation tasks. Inspired by existing work on combining 2D and 3D computations for volumetric data, both in deep learning 5,6 and more generally 26 , we experimented with combinations of 2D and 3D neural modules to trade off between computational efficiency and spatial context. The highest-performing network architecture in this paper, 2D-3D+3x3x3, is a composition of a 2D U-net-style encoder-decoder and 3D convolutional spatial pyramid, with additional 3x3x3 convolutions at the beginning of convolution blocks in the encoder-decoder.…”

Section: /16mentioning

confidence: 99%

Dense cellular segmentation for EM using 2D-3D neural network ensembles

Guay

Emam

Anderson

et al. 2020

Preprint

View full text Add to dashboard Cite

Modern biological electron microscopy produces nanoscale images from biological samples of unprecedented volume, and researchers now face the problem of making use of the data. Image segmentation has played a fundamental role in EM image analysis for decades, but challenges from biological EM have spurred interest and rapid advances in computer vision for automating the segmentation process. In this paper, we demonstrate dense cellular segmentation as a method for generating rich 3D models of tissues and their constituent cells and organelles from scanning electron microscopy images. We describe how to use ensembles of 2D-3D neural networks to compute dense cellular segmentations of cells and organelles inside two human platelet tissue samples. We conclude by discussing ongoing challenges for realizing practical dense cellular segmentation algorithms. The data and code used in this paper, as well as example notebooks, are available at leapmanlab. github.io/dense-cell.Biomedical researchers use electron microscopy (EM) to image cells, organelles, and their constituents at the nanoscale. The serial block-face scanning electron microscope (SBF-SEM) 1 uses automated serial sectioning techniques to rapidly produce gigavoxel image volumes and beyond. This rapid growth in throughput challenges traditional image analytic workflows for EM, which rely on trained humans to identify salient image features. Highthroughput EM offers to revolutionize systems biology by providing nanoscale structural detail across macroscopic tissue regions, but analyses of such datasets in their entirety will be infeasibly expensive and time-consuming until analytic bottlenecks are automated. A fundamental component of common EM image anaylsis workflows is segmentation, which groups image voxels together into labeled regions that correspond to image content. For semantic segmentation, each voxel is assigned an object classification label, such as cell or mitochondrion. In this paper we introduce a dense cellular segmentation task, illustrated in Figure 1, which seeks to segment the entirety of an image volume with multiple, detailed class labels. Dense semantic segmentation is vital for systems biologists seeking to create semantically-rich 3D models of cells and subcellular structure interconnected within tissue environments. Manually performing dense segmentation tasks for EM volumes is tedious and infeasible at scale for new highthroughput microscopes. However, automating dense segmentation for EM is challenging due to the image complexity of biological structures at the nanoscale. An image with little noise and high contrast between features may be accurately segmented with simple methods such as thresholding 2 , while accurate segmentation of complicated images with multiscale features, noise, and textural content remains an open problem for many tasks of interest to biomedicine. Current state-of-the-art computational solutions use convolutional neural network architectures to solve problems on a per-task basis. The field of connectomics h...

show abstract

2D-3D Fully Convolutional Neural Networks for Cardiac MR Segmentation

Cited by 79 publications

References 11 publications

Deep Learning for Cardiac Image Segmentation: A Review

Deep Learning for Cardiac Image Segmentation: A Review

An iterative multi‐path fully convolutional neural network for automatic cardiac segmentation in cine MR images

Dense cellular segmentation for EM using 2D-3D neural network ensembles

Contact Info

Product

Resources

About