Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks

Abstract

Automatic segmentation of the liver and hepatic lesions is an important step towards deriving quantitative biomarkers for accurate clinical diagnosis and computer-aided decision support systems. This paper presents a method to automatically segment liver and lesions in CT and MRI abdomen images using cascaded fully convolutional neural networks (CFCNs) enabling the segmentation of large-scale medical trials and quantitative image analyses. We train and cascade two FCNs for the combined segmentation of the liver and its lesions. As a first step, we train an FCN to segment the liver as ROI input for a second FCN. The second FCN solely segments lesions within the predicted liver ROIs of step 1. CFCN models were trained on an abdominal CT dataset comprising 100 hepatic tumor volumes. Validation results on further datasets show that CFCN-based semantic liver and lesion segmentation achieves Dice scores over 94% for the liver with computation times below 100s per volume. We further experimentally demonstrate the robustness of the proposed method on 38 MRI liver tumor volumes and the public 3DIRCAD dataset.

Joint Liver and Hepatic Lesion Segmentation in MRI using a Hybrid CNN with Transformer Layers

“…So far, to the best of the authors' knowledge, there are hardly any published works focusing on deep learningbased hepatic lesion segmentation in MR imaging, except the work of Christ et al [7], which included MRI data in their otherwise CT-based study. The authors utilized an Unetstyle fully convolutional cascaded neural network with a 3D Conditional Random Field (CRF) for the segmentation of the liver and subsequently using the resulting liver mask as an input for the following lesion segmentation.…”

“…Compared to our proposed SWTR-Unet, both segmentations of liver and lesions yielded inferior results of approximately 11%. Furthermore, Christ et al [7] separated both segmentation task, which results in a dependency on the quality of the preceding liver segmentation step with potential implications with respect to the segmentation accuracy of near-surface tumours in particular. Overall, it could be concluded, that the solely convolutional-based architecture style has its limitations regarding the huge shape, size and location variety, which may not be adequately captured due to the focus of merely local information and features, respectively.…”

“…In recent years various works focused on CT-based liver and liver lesion segmentation, especially since the liver tumor segmentation (LiTS) challenge publicly provided patient CT scans, which are otherwise hard to compromise in reasonable numbers. In terms of MRI-based hepatic lesions segmentation, there currently is, to the best of the authors' knowledge, only the work of Christ et al [7] of direct relevance. They presented a fully automatic approach consisting of a cascaded convolutional neural network (CNN), that first segments the liver outline and subsequently the hepatic lesions within the resulting liver mask images.…”

“…They presented a fully automatic approach consisting of a cascaded convolutional neural network (CNN), that first segments the liver outline and subsequently the hepatic lesions within the resulting liver mask images. Christ et al [7] applied their two-step strategy to both, liver CT and a few MR images and achieved averaged liver Dice similarity scores of 94% (CT) and 87% (MRI), as well as lesion Dice similarity scores on average of 61% (CT) and 69.7% (MRI). Their used dataset was comprised of 100 CT patient scans and thereof 20 validation cases as well as 31 MRI scans without any details about the validation set size.…”

Inconsistency-aware Uncertainty Estimation for Semi-supervised Medical Image Segmentation

“…In terms of the segmentation under supervised setting, among the current SOTA methods, the most popular framework is the fully convolutional network (FCN) [35] based encoder-decoder architecture with the representative model like U-Net [45]. So far, to make the conventional encoder-decoder architecture more effective and robust, researchers have made great efforts in the following three directions: 1) developing novel structures, including the 3D structure, recurrent neural network (RNN) based model and cascaded framework [39] [7] [4] [48] [12] [62], 2) designing novel network blocks, including attention mechanism, dense connection, inception or multi-scale fusion [8] [65] [11] [57] [41], and 3) utilizing sophisticated loss functions [58] [10] [26] [61], significantly improving segmentation accuracy.…”