Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences

Chen, Qin; Bai, Wenjia; Schlemper, Jo; Petersen, Steffen E.; Piechnik, Stefan K.; Neubauer, Stefan; Rueckert, Daniel

doi:10.1007/978-3-030-00934-2_53

Cited by 119 publications

(127 citation statements)

References 16 publications

(27 reference statements)

Supporting

Mentioning

125

Contrasting

Order By: Relevance

“…These networks can also be applied to leveraging information across different temporal frames in the cardiac cycle to improve spatial and temporal consistency of segmentation results (Yan et al, 2018;Du et al, 2019;Qin et al, 2018a;Wolterink et al, 2017c).…”

Section: Ventricle Segmentationmentioning

confidence: 99%

“…One approach is to estimate full labels on unlabeled or weakly labeled images for further training. For example, Bai et al (2018b); Qin et al (2018a) utilized motion information to propagate labels from labeled frames to unlabeled frames in a cardiac cycle whereas Bai et al (2017); Can et al (2018) applied the expectation maximization (EM) algorithm to predict and refine the estimated labels recursively. Others have explored different approaches to regularize the network when training on unlabeled images, applying multitask learning , or global constraints (Kervadec et al, 2019).…”

Section: Scarcity Of Labelsmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning for Cardiac Image Segmentation: A Review

Chen

Qiu

et al. 2020

Front. Cardiovasc. Med.

Self Cite

562

383

View full text Add to dashboard Cite

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research.

show abstract

Section: Ventricle Segmentationmentioning

confidence: 99%

Section: Scarcity Of Labelsmentioning

confidence: 99%

Deep Learning for Cardiac Image Segmentation: A Review

Chen

Qiu

et al. 2020

Front. Cardiovasc. Med.

Self Cite

562

383

View full text Add to dashboard Cite

show abstract

“…The world surrounding us is typically affected by smooth temporal variations and is known that temporal consistency plays a key role for the development of invariant representations in biological vision [17]. However, despite that temporal correlations have been used to learn/propagate segmentations in medical imaging [1,12], their use as a learning signal to improve representations remains unexplored. To the best of our knowledge, this is the first work to use spatiotemporal dynamics to improve disentangled representations in cardiac imaging.…”

Section: Learning Good Representations With Temporal Conditioningmentioning

confidence: 99%

Temporal Consistency Objectives Regularize the Learning of Disentangled Representations

Valvano

Chartsias

Leo

et al. 2019

Domain Adaptation and Representation Transfer and Medical Image Learning With Less Labels and Imperfect Data

View full text Add to dashboard Cite

There has been an increasing focus in learning interpretable feature representations, particularly in applications such as medical image analysis that require explainability, whilst relying less on annotated data (since annotations can be tedious and costly). Here we build on recent innovations in style-content representations to learn anatomy, imaging characteristics (appearance) and temporal correlations. By introducing a self-supervised objective of predicting future cardiac phases we improve disentanglement. We propose a temporal transformer architecture that given an image conditioned on phase difference, it predicts a future frame. This forces the anatomical decomposition to be consistent with the temporal cardiac contraction in cine MRI and to have semantic meaning with less need for annotations. We demonstrate that using this regularization, we achieve competitive results and improve semisupervised segmentation, especially when very few labelled data are available. Specifically, we show Dice increase of up to 19% and 7% compared to supervised and semi-supervised approaches respectively on the ACDC dataset. Code is available at: https://github.com/gvalvano/sdtnet.

show abstract

“…In this paper, we propose to learn a joint deep learning network for cardiac motion estimation and segmentation directly from undersampled cardiac MR data, bypassing the MR reconstruction process. In particular, we extend the joint model proposed in [6] which consists of an unsupervised cardiac motion estimation branch and a weakly-supervised segmentation branch, where the two tasks share the same feature encoder. We investigate the network's capability of predicting motion estimation and segmentation maps simultaneously and directly from undersampled cardiac MR data.…”

Section: Introductionmentioning

confidence: 99%

“…We investigate the network's capability of predicting motion estimation and segmentation maps simultaneously and directly from undersampled cardiac MR data. The problem is formulated by incorporating supervision from fully sampled MR image pairs in addition to the composite loss function as proposed in [6]. Simulation experiments have been performed on 220 subjects under different acceleration factors with radial undersampling patterns.…”

Section: Introductionmentioning

confidence: 99%

Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image

Chen

Bai

Schlemper

et al. 2018

Machine Learning for Medical Image Reconstruction

Self Cite

View full text Add to dashboard Cite

Accelerating the acquisition of magnetic resonance imaging (MRI) is a challenging problem, and many works have been proposed to reconstruct images from undersampled k-space data. However, if the main purpose is to extract certain quantitative measures from the images, perfect reconstructions may not always be necessary as long as the images enable the means of extracting the clinically relevant measures. In this paper, we work on jointly predicting cardiac motion estimation and segmentation directly from undersampled data, which are two important steps in quantitatively assessing cardiac function and diagnosing cardiovascular diseases. In particular, a unified model consisting of both motion estimation branch and segmentation branch is learned by optimising the two tasks simultaneously. Additional corresponding fully-sampled images are incorporated into the network as a parallel sub-network to enhance and guide the learning during the training process. Experimental results using cardiac MR images from 220 subjects show that the proposed model is robust to undersampled data and is capable of predicting results that are close to that from fully-sampled ones, while bypassing the usual image reconstruction stage.

show abstract

Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences

Cited by 119 publications

References 16 publications

Deep Learning for Cardiac Image Segmentation: A Review

Deep Learning for Cardiac Image Segmentation: A Review

Temporal Consistency Objectives Regularize the Learning of Disentangled Representations

Joint Motion Estimation and Segmentation from Undersampled Cardiac MR Image

Contact Info

Product

Resources

About