DS$$^3$$-Net: Difficulty-Perceived Common-to-T1ce Semi-supervised Multimodal MRI Synthesis Network

Huang, Ziqi; Lin, Li; Cheng, Pujin; Pan, Kai; Tang, Xiaoying

doi:10.1007/978-3-031-16446-0_54

Cited by 4 publications

(8 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GAN [45] has become the mainstay of medical image synthesis, with common applications in intra-modality augmentation [46], cross-domain image-to-image translation [47], quality enhancement [48], missing modality generation [49], etc. Below we briefly review previous works on retinal image synthesis, the topic of which is relevant to our work.…”

Section: Medical Image Synthesismentioning

confidence: 99%

YoloCurvSeg: You Only Label One Noisy Skeleton for Vessel-style Curvilinear Structure Segmentation

Li¹,

Peng²,

He³

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Weakly-supervised learning (WSL) has been proposed to alleviate the conflict between data annotation cost and model performance through employing sparsely-grained (i.e., point-, box-, scribble-wise) supervision and has shown promising performance, particularly in the image segmentation field. However, it is still a very challenging problem due to the limited supervision, especially when only a small number of labeled samples are available. Additionally, almost all existing WSL segmentation methods are designed for star-convex structures which are very different from curvilinear structures such as vessels and nerves. In this paper, we propose a novel sparsely annotated segmentation framework for curvilinear structures, named YoloCurvSeg, based on image synthesis. A background generator delivers image backgrounds that closely match real distributions through inpainting dilated skeletons. The extracted backgrounds are then combined with randomly emulated curves generated by a Space Colonization Algorithm-based foreground generator and through a multilayer patch-wise contrastive learning synthesizer. In this way, a synthetic dataset with both images and curve segmentation labels is obtained, at the cost of only one or a few noisy skeleton annotations.</p> <p>Finally, a segmenter is trained with the generated dataset and possibly an unlabeled dataset. The proposed YoloCurvSeg is evaluated on four publicly available datasets (OCTA500, CORN, DRIVE and CHASEDB1) and the results show that YoloCurvSeg outperforms state-of-the-art WSL segmentation methods by large margins. With only one noisy skeleton annotation (respectively 0.14%, 0.03%, 1.40%, and 0.65% of the full annotation), YoloCurvSeg achieves more than 97% of the fully-supervised performance on each dataset. Code and datasets will be released at https://github.com/llmir/YoloCurvSeg.</p>

show abstract

Section: Medical Image Synthesismentioning

confidence: 99%

YoloCurvSeg: You Only Label One Noisy Skeleton for Vessel-style Curvilinear Structure Segmentation

Li¹,

Peng²,

He³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…MRI synthesis approaches based on deep learning currently serve as an emerging field of research in neuro‐oncology 8,9 . In particular, various deep learning‐based approaches have been investigated for contrast‐enhanced MRI synthesis toward reduction 7,10–13 or even elimination 8,14–22 of gadolinium contrast agents in glioma patients. The former involves methods that propose the synthesis of full‐dose contrast‐enhanced images from their low‐dose counterparts (e.g., 10% low‐dose).…”

Section: Introductionmentioning

confidence: 99%

“…Current research focuses on the complete elimination of gadolinium contrast agent administration via developing methods utilizing deep learning techniques to generate contrast‐enhanced MR synthetic images from contrast‐free ones. The proposed deep learning networks can be grouped into convolutional neural networks (CNNs), 13,16,22 U‐Net, 16,18 Bayesian U‐Net, 14 and generative adversarial networks 8,14–16 – 17,19,21 . Some of them use multiple MRI sequences as input, 8,15,16,19,21,22 while others utilize just a single contrast‐free MRI sequence 17,18 …”

Section: Introductionmentioning

confidence: 99%

“…The proposed deep learning networks can be grouped into convolutional neural networks (CNNs), 13,16,22 U‐Net, 16,18 Bayesian U‐Net, 14 and generative adversarial networks 8,14–16 – 17,19,21 . Some of them use multiple MRI sequences as input, 8,15,16,19,21,22 while others utilize just a single contrast‐free MRI sequence 17,18 …”

Section: Introductionmentioning

confidence: 99%

“…Although the reported results have demonstrated the feasibility of these models for contrast‐enhanced MRI synthesis, the datasets used for training and evaluation of the models are relatively small, 8,10,13–15,18,19,20,21,22 containing no more than 800 patients except one study 16 . Moreover, existing work has insufficient performance on tumor/lesion regions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Contrast‐enhanced MRI synthesis using dense‐dilated residual convolutions based 3D network toward elimination of gadolinium in neuro‐oncology

Osman

Tamam

2023

J Applied Clin Med Phys

View full text Add to dashboard Cite

Recent studies have raised broad safety and health concerns about using of gadolinium contrast agents during magnetic resonance imaging (MRI) to enhance identification of active tumors. In this paper, we developed a deep learning‐based method for three‐dimensional (3D) contrast‐enhanced T1‐weighted (T1) image synthesis from contrast‐free image(s). The MR images of 1251 patients with glioma from the RSNA‐ASNR‐MICCAI BraTS Challenge 2021 dataset were used in this study. A 3D dense‐dilated residual U‐Net (DD‐Res U‐Net) was developed for contrast‐enhanced T1 image synthesis from contrast‐free image(s). The model was trained on a randomly split training set (n = 800) using a customized loss function and validated on a validation set (n = 200) to improve its generalizability. The generated images were quantitatively assessed against the ground‐truth on a test set (n = 251) using the mean absolute error (MAE), mean‐squared error (MSE), peak signal‐to‐noise ratio (PSNR), structural similarity (SSIM), normalized mutual information (NMI), and Hausdorff distance (HDD) metrics. We also performed a qualitative visual similarity assessment between the synthetic and ground‐truth images. The effectiveness of the proposed model was compared with a 3D U‐Net baseline model and existing deep learning‐based methods in the literature. Our proposed DD‐Res U‐Net model achieved promising performance for contrast‐enhanced T1 synthesis in both quantitative metrics and perceptual evaluation on the test set (n = 251). Analysis of results on the whole brain region showed a PSNR (in dB) of 29.882 ± 5.924, a SSIM of 0.901 ± 0.071, a MAE of 0.018 ± 0.013, a MSE of 0.002 ± 0.002, a HDD of 2.329 ± 9.623, and a NMI of 1.352 ± 0.091 when using only T1 as input; and a PSNR (in dB) of 30.284 ± 4.934, a SSIM of 0.915 ± 0.063, a MAE of 0.017 ± 0.013, a MSE of 0.001 ± 0.002, a HDD of 1.323 ± 3.551, and a NMI of 1.364 ± 0.089 when combining T1 with other MRI sequences. Compared to the U‐Net baseline model, our model revealed superior performance. Our model demonstrated excellent capability in generating synthetic contrast‐enhanced T1 images from contrast‐free MR image(s) of the whole brain region when using multiple contrast‐free images as input. Without incorporating tumor mask information during network training, its performance was inferior in the tumor regions compared to the whole brain which requires further improvements to replace the gadolinium administration in neuro‐oncology.

show abstract

Knowledge‐driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un‐supervised learning

Wang,

Wu,

Jia

et al. 2024

Magnetic Resonance in Med

View full text Add to dashboard Cite

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics‐based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data‐driven approaches. Our review will introduce the significant challenges faced by such knowledge‐driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi‐supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

show abstract

DS$$^3$$-Net: Difficulty-Perceived Common-to-T1ce Semi-supervised Multimodal MRI Synthesis Network

Cited by 4 publications

References 26 publications

YoloCurvSeg: You Only Label One Noisy Skeleton for Vessel-style Curvilinear Structure Segmentation

YoloCurvSeg: You Only Label One Noisy Skeleton for Vessel-style Curvilinear Structure Segmentation

Contrast‐enhanced MRI synthesis using dense‐dilated residual convolutions based 3D network toward elimination of gadolinium in neuro‐oncology

Knowledge‐driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un‐supervised learning

Contact Info

Product

Resources

About