Full Quantification of Left Ventricle via Deep Multitask Learning Network Respecting Intra- and Inter-Task Relatedness

Abstract

Cardiac left ventricle (LV) quantification is among the most clinically important tasks for identification and diagnosis of cardiac diseases, yet still a challenge due to the high variability of cardiac structure and the complexity of temporal dynamics. Full quantification, i.e., to simultaneously quantify all LV indices including two areas (cavity and myocardium), six regional wall thicknesses (RWT), three LV dimensions, and one cardiac phase, is even more challenging since the uncertain relatedness intra and inter each type of indices may hinder the learning procedure from better convergence and generalization. In this paper, we propose a newly-designed multitask learning network (FullLVNet), which is constituted by a deep convolution neural network (CNN) for expressive feature embedding of cardiac structure; two followed parallel recurrent neural network (RNN) modules for temporal dynamic modeling; and four linear models for the final estimation. During the final estimation, both intra-and inter-task relatedness are modeled to enforce improvement of generalization: 1) respecting intra-task relatedness, group lasso is applied to each of the regression tasks for sparse and common feature selection and consistent prediction; 2) respecting inter-task relatedness, three phase-guided constraints are proposed to penalize violation of the temporal behavior of the obtained LV indices. Experiments on MR sequences of 145 subjects show that FullLVNet achieves high accurate prediction with our intra-and inter-task relatedness, leading to MAE of 190mm 2 , 1.41mm, 2.68mm for average areas, RWT, dimensions and error rate of 10.4% for the phase classification. This endows our method a great potential in comprehensive clinical assessment of global, regional and dynamic cardiac function.

Recurrent Neural Networks for Aortic Image Sequence Segmentation with Sparse Annotations

“…In some works, RNN was used to model the spatial dependency in static images [4][5][6], such as the inter-slice dependency in anisotropic images [4,5]. In other works, RNN was used to model the temporal dependency in image sequences [7][8][9]. For example, Kong et al used RNN to model the temporal dependency in cardiac MR image sequences and to predict the cardiac phase for each time frame [7].…”

Left Ventricle Quantification Through Spatio-Temporal CNNs

“…We use a first CNN (referred as encoder-CNN in Figures 2 and 3) to extract informative features from individual slices. Inspired by [11], we designed the per-slice encoding phase using a two-layers CNN where the convolutional and pooling kernels are of size 5x5, instead of the frequently used 3x3, to introduce more shift invariance (see Figure 3 for more details). We use ReLU activation function and batch normalization to alleviate the training process.…”

Multi-estimator Full Left Ventricle Quantification Through Ensemble Learning

“…This category of methods is advantages as the results (usually as segmentation masks) are easier to interpret which can provide more insight into the analytic procedure. The latter category of direct estimation-based methods [5,6,7,8] perform estimation of the cardiac parameters directly from the image and/or image features, without relying on explicit segmentation procedure. It is advantages as the training error can be directly back-propagated to the feature selection and regression process, thus normally resulting in higher accuracy.…”