Probabilistic Motion Modeling from Medical Image Sequences: Application to Cardiac Cine-MRI

Abstract

We propose to learn a probabilistic motion model from a sequence of images. Besides spatio-temporal registration, our method offers to predict motion from a limited number of frames, useful for temporal super-resolution. The model is based on a probabilistic latent space and a novel temporal dropout training scheme. This enables simulation and interpolation of realistic motion patterns given only one or any subset of frames of a sequence. The encoded motion also allows to be transported from one subject to another without the need of inter-subject registration. An unsupervised generative deformation model is applied within a temporal convolutional network which leads to a diffeomorphic motion model -encoded as a low-dimensional motion matrix. Applied to cardiac cine-MRI sequences, we show improved registration accuracy and spatio-temporally smoother deformations compared to three stateof-the-art registration algorithms. Besides, we demonstrate the model's applicability to motion transport by simulating a pathology in a healthy case. Furthermore, we show an improved motion reconstruction from incomplete sequences compared to linear and cubic interpolation.

CinE caRdiac magneTic resonAnce to predIct veNTricular arrhYthmia (CERTAINTY)

“…In this work, the above-mentioned cine CMR dataset was used to train two independent neural networks. First, a probabilistic encoder-decoder neural network 14 was trained to extract cardiac structure and function features from 4cv cine CMR in a form of cine fingerprint in a fully unsupervised fashion ( Cine Fingerprint Extractor , Fig. 1 A).…”

“…Recently, AI has been shown to predict personalized prognosis, such as individual responses to lung cancer therapy 12 and survival for patients with pulmonary hypertension 13 . While traditional machine learning approaches rely on handcrafted, previously recognized features extracted from medical images, AI can also automatically generate a patient-specific fingerprint containing inherent features of cardiac structure and function from cine CMR 14 in an unsupervised fashion 15 .…”

Deep learning in medical image registration

“…The primary advantage of DLIR methods is their ability to compensate for soft tissue and patient motion in real-time, setting them apart from iterative traditional registration approaches. For instance, Krebs et al [101] designed an unsupervised generative deformation model within a temporal convolutional network to learn a probabilistic motion model from a sequence of images, which could be applied for both cardiac cine MRI spatio-temporal registration and motion analysis. Such an approach could be used for real-time cardiac motion analysis, providing the basis for discovery of novel motion-based disease biomarkers.…”

“…Several brain MRI datasets are also utilised for developing multi-modal image registration methods [14,38], with T1W and T2W modalities available in most brain MRI datasets. Apart from neuroimaging, cine MRI is the primary modality used for cardiac image registration and cardiac motion estimation [64,101], with two available public datasets, Sunnybrook Cardiac Data (SCD) [104] and Automatic Cardiac Diagnosis Challenge (ACDC) [105].…”

Learning a Generative Motion Model from Image Sequences based on a Latent Motion Matrix

“…• An unsupervised probabilistic motion model learned from medical image sequences • A conditional VAE model trained with a novel Gaussian process prior and self-supervised temporal dropout using temporal convolutional networks • Demonstration of cardiac motion tracking, simulation, transport and temporal super-resolution This paper extends our preliminary conference paper [35] by replacing the standard unit Gaussian of the CVAE with a novel Gaussian process Prior. We add detailed derivations of the motion model and show improved tracking accuracy and temporal smoothness.…”