Joint reconstruction and segmentation in undersampled 3D knee MRI combining shape knowledge and deep learning

Abstract

Task-adapted image reconstruction methods using end-to-end trainable
neural networks (NNs) have been proposed to optimize reconstruction for subsequent
processing tasks, such as segmentation. However, their training typically requires
considerable hardware resources and thus, only relatively simple building blocks,
e.g. U-Nets, are typically used, which, albeit powerful, do not integrate model-
specific knowledge. In this work, we extend an end-to-end trainable task-adapted
image reconstruction method for a clinically realistic reconstruction and segmentation
problem of bone and cartilage in 3D knee MRI by incorporating statistical shape models
(SSMs). The SSMs model the prior information and help to regularize the segmentation
maps as a final post-processing step. We compare the proposed method to a 
simultaneous multitask learning approach for image reconstruction and
segmentation (MTL) and to a complex SSMs-informed segmentation pipeline (SIS).
Our experiments show that the combination of joint end-to-end training and SSMs
to further regularize the segmentation maps obtained by MTL highly improves the
results, especially in terms of mean and maximal surface errors. In particular, we
achieve the segmentation quality of SIS and, at the same time, a substantial model
reduction that yields a five-fold decimation in model parameters and a computational
speedup of an order of magnitude. Remarkably, even for undersampling factors of up to
R = 8, the obtained segmentation maps are of comparable quality to those obtained
by SIS from ground-truth images.