Contextual loss functions for optimization of convolutional neural networks generating pseudo CTs from MRI

Stralen, Marijn van; Zhou, Yucan; Wozny, Phillip; Seevinck, Peter R.; Loog, Marco

doi:10.1117/12.2293749

Cited by 7 publications

(6 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[20][21][22] In parallel to the use of specialized and combined MR images, processing techniques have also expanded to involve statistical 17 and machine learning models, 14 with recent advances in deep learning. 6,10,12,[23][24][25][26][27][28][29][30][31] Recent studies 23,25,31 have shown that deep learning-based models generate equivalent or better sCTs than more conventional methods. Particularly encouraging results were obtained for radiotherapy treatment planning purposes 6,7 and PET/MR 10,11,24,29 while using automated workflows.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning–based MR‐to‐CT synthesis: The influence of varying gradient echo–based MR images as input channels

Florkow

Zijlstra

Willemsen

et al. 2019

Magnetic Resonance in Med

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

“…In parallel to the use of specialized and combined MR images, processing techniques have also expanded to involve statistical and machine learning models, with recent advances in deep learning . Recent studies have shown that deep learning–based models generate equivalent or better sCTs than more conventional methods.…”

Section: Introductionmentioning

confidence: 99%

Deep learning–based MR‐to‐CT synthesis: The influence of varying gradient echo–based MR images as input channels

Florkow

Zijlstra

Willemsen

et al. 2019

Magnetic Resonance in Med

Self Cite

View full text Add to dashboard Cite

show abstract

“…Specifically, the perceptual loss aims to synthesize T1WI planning CTs using deep learning (DL)-based frameworks, U-Net and CycleGAN (24). The contextual loss function has been used in a fully convolutional neural network (FCN) to generate pseudo-CTs from MRI, which confirms that it can improve the predicted performance of the CNN without changing the network architecture (26).…”

Section: Introductionmentioning

confidence: 94%

“…Contextual loss calculates the similarity between the real MR images I MR and synthetic MR images Syn MR (I CT ) using the following mathematical formulas (23,26): First, d ij is the raw distance, which represents the cosine distance between x i and y j , ( ) ( )…”

Section: Contextual Lossmentioning

confidence: 99%

Synthesis of magnetic resonance images from computed tomography data using convolutional neural network with contextual loss function

Li¹,

Huang²,

Zhang³

et al. 2022

Quant Imaging Med Surg

View full text Add to dashboard Cite

Background: Magnetic resonance imaging (MRI) images synthesized from computed tomography (CT) data can provide more detailed information on pathological structures than that of CT data alone; thus, the synthesis of MRI has received increased attention especially in medical scenarios where only CT images are available. A novel convolutional neural network (CNN) combined with a contextual loss function was proposed for synthesis of T1-and T2-weighted images (T1WI and T2WI) from CT data.Methods: A total of 5,053 and 5,081 slices of T1WI and T2WI, respectively were selected for the dataset of CT and MRI image pairs. Affine registration, image denoising, and contrast enhancement were done on the aforementioned multi-modality medical image dataset comprising T1WI, T2WI, and CT images of the brain. A deep CNN was then proposed by modifying the ResNet structure to constitute the encoder and decoder of U-Net, called double ResNet-U-Net (DRUNet). Three different loss functions were utilized to optimize the parameters of the proposed models: mean squared error (MSE) loss, binary crossentropy (BCE) loss, and contextual loss. Statistical analysis of the independent-sample t-test was conducted by comparing DRUNets with different loss functions and different network layers. Results: DRUNet-101 with contextual loss yielded higher values of peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and Tenengrad function (i.e., 34.25±2.06, 0.97±0.03, and 17.03±2.75 for T1WI and 33.50±1.08, 0.98±0.05, and 19.76±3.54 for T2WI respectively). The results were statistically significant at P<0.001 with a narrow confidence interval of difference, indicating the superiority of DRUNet-101 with contextual loss. In addition, both image zooming and difference maps presented for the final synthetic MR images visually reflected the robustness of DRUNet-101 with contextual loss. The visualization of convolution filters and feature maps showed that the proposed model can generate synthetic MR images with high-frequency information. Conclusions: The results demonstrated that DRUNet-101 with contextual loss function provided better high-frequency information in synthetic MR images compared with the other two functions. The proposed DRUNet model has a distinct advantage over previous models in terms of PSNR, SSIM, and Tenengrad score.Overall, DRUNet-101 with contextual loss is recommended for synthesizing MR images from CT scans.

show abstract

“…To objectively assess the synthesized image quality, all per-class images obtained from the selected model generators were quantitatively evaluated using the mean square error (MSE), structural similarity (SSIM) index, and the peak signal-to-noise ratio (PSNR). [25][26][27] The MSE represents the cumulative squared error between the synthesized and original images. The SSIM index measures the structural information similarity between images, where 0 indicates no similarity and 1 indicates complete similarity.…”

Section: Objective Evaluation Metrics For Synthesized Imagesmentioning

confidence: 99%

Pix2pix Conditional Generative Adversarial Networks for Scheimpflug Camera Color-Coded Corneal Tomography Image Generation

Abdelmotaal

Abdou

Omar

et al. 2021

Trans. Vis. Sci. Tech.

View full text Add to dashboard Cite

To assess the ability of pix2pix conditional generative adversarial network (pix2pix cGAN) to create plausible synthesized Scheimpflug camera color-coded corneal tomography images based upon a modest-sized original dataset to be used for image augmentation during training a deep convolutional neural network (DCNN) for classification of keratoconus and normal corneal images.Methods: Original images of 1778 eyes of 923 nonconsecutive patients with or without keratoconus were retrospectively analyzed. Images were labeled and preprocessed for use in training the proposed pix2pix cGAN. The best quality synthesized images were selected based on the Fréchet inception distance score, and their quality was studied by calculating the mean square error, structural similarity index, and the peak signal-to-noise ratio. We used original, traditionally augmented original and synthesized images to train a DCNN for image classification and compared classification performance metrics. Results:The pix2pix cGAN synthesized images showed plausible subjectively and objectively assessed quality. Training the DCNN with a combination of real and synthesized images allowed better classification performance compared with training using original images only or with traditional augmentation. Conclusions:Using the pix2pix cGAN to synthesize corneal tomography images can overcome issues related to small datasets and class imbalance when training computeraided diagnostic models.Translational Relevance: Pix2pix cGAN can provide an unlimited supply of plausible synthetic Scheimpflug camera color-coded corneal tomography images at levels useful for experimental and clinical applications.

show abstract

Contextual loss functions for optimization of convolutional neural networks generating pseudo CTs from MRI

Cited by 7 publications

References 9 publications

Deep learning–based MR‐to‐CT synthesis: The influence of varying gradient echo–based MR images as input channels

Deep learning–based MR‐to‐CT synthesis: The influence of varying gradient echo–based MR images as input channels

Synthesis of magnetic resonance images from computed tomography data using convolutional neural network with contextual loss function

Pix2pix Conditional Generative Adversarial Networks for Scheimpflug Camera Color-Coded Corneal Tomography Image Generation

Contact Info

Product

Resources

About