Extended Modality Propagation: Image Synthesis of Pathological Cases

Cordier, Nicolas; Delingette, Hervé; Le, Matthieu; Ayache, Nicholas

doi:10.1109/tmi.2016.2589760

Cited by 41 publications

(25 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The learned sparse combinations are then applied to estimate patches in y 2 from patches in y 1 . To improve matching of patches across domains, generative models were also proposed that use multi-scale patches and tissue segmentation labels [16], [18]. Instead of focusing on linear models, recent studies aimed to learn more general non-linear mappings that express individual voxels in y 1 in terms of patches in x 1 , and then predict y 2 from x 2 based on these mappings.…”

Section: Introductionmentioning

confidence: 99%

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Dar

Yurt

Karacan

et al. 2019

IEEE Trans. Med. Imaging

402

295

View full text Add to dashboard Cite

Acquiring images of the same anatomy with multiple different contrasts increases the diversity of diagnostic information available in an MR exam. Yet, the scan time limitations may prohibit the acquisition of certain contrasts, and some contrasts may be corrupted by noise and artifacts. In such cases, the ability to synthesize unacquired or corrupted contrasts can improve diagnostic utility. For multi-contrast synthesis, the current methods learn a nonlinear intensity transformation between the source and target images, either via nonlinear regression or deterministic neural networks. These methods can, in turn, suffer from the loss of structural details in synthesized images. Here, in this paper, we propose a new approach for multi-contrast MRI synthesis based on conditional generative adversarial networks. The proposed approach preserves intermediate-to-high frequency details via an adversarial loss, and it offers enhanced synthesis performance via pixel-wise and perceptual losses for registered multi-contrast images and a cycle-consistency loss for unregistered images. Information from neighboring cross-sections are utilized to further improve synthesis quality. Demonstrations on T 1-and T 2-weighted images from healthy subjects and patients clearly indicate the superior performance of the proposed approach compared to the previous state-of-the-art methods. Our synthesis approach can help improve the quality and versatility Manuscript

show abstract

Section: Introductionmentioning

confidence: 99%

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Dar

Yurt

Karacan

et al. 2019

IEEE Trans. Med. Imaging

402

295

View full text Add to dashboard Cite

show abstract

“…Another common approach to MRI synthesis is an intensity transformation-based method. [13][14][15][16][17][18][19] This method generates a synthetic image as an optimal linear combination of patches from the atlas related to the input patches. 13 To improve the performance of the patch matching algorithm, sparse dictionary reconstruction using multiple patches, 14 and a method employing multi-scale patches with tissue masks have been suggested.…”

Section: Introductionmentioning

confidence: 99%

“…13 To improve the performance of the patch matching algorithm, sparse dictionary reconstruction using multiple patches, 14 and a method employing multi-scale patches with tissue masks have been suggested. 15,16 However, these methods take a long time to find the nearest neighbor patches for each test voxel. A method using non-linear regression models produced better results than the previous methods in terms of time and image quality.…”

Section: Introductionmentioning

confidence: 99%

Deep‐learned short tau inversion recovery imaging using multi‐contrast MR images

Kim

Jang

et al. 2020

Magnetic Resonance in Med

View full text Add to dashboard Cite

Purpose To generate short tau, or short inversion time (TI), inversion recovery (STIR) images from three multi‐contrast MR images, without additional scanning, using a deep neural network. Methods For simulation studies, we used multi‐contrast simulation images. For in‐vivo studies, we acquired knee MR images including 288 slices of T1‐weighted (T1‐w), T2‐weighted (T2‐w), gradient‐recalled echo (GRE), and STIR images taken from 12 healthy volunteers. Our MR image synthesis method generates a new contrast MR image from multi‐contrast MR images. We used a deep neural network to identify the complex relationships between MR images that show various contrasts for the same tissues. Our contrast‐conversion deep neural network (CC‐DNN) is an end‐to‐end architecture that trains the model to create one image from three (T1‐w, T2‐w, and GRE images). We propose a new loss function to take into account intensity differences, misregistration, and local intensity variations. The CC‐DNN‐generated STIR images were evaluated with four quantitative evaluation metrics, including mean squared error, peak signal‐to‐noise ratio (PSNR), structural similarity (SSIM), and multi‐scale SSIM (MS‐SSIM). Furthermore, a subjective evaluation was performed by musculoskeletal radiologists. Results Our method showed improved results in all quantitative evaluations compared with other methods and received the highest scores in subjective evaluations by musculoskeletal radiologists. Conclusion This study suggests the feasibility of our method for generating STIR sequence images without additional scanning that offered a potential alternative to the STIR pulse sequence when additional scanning is limited or STIR artifacts are severe.

show abstract

“…In these and other applications, it would be desirable to have a cross-modality image synthesis method that can generate the target modality images from the source modality scans. The ability to synthesize different modalities of the same anatomy can benefit various practical image analysis tasks including multi-modal registration [6,7], segmentation [8], and atlas construction [9,10].…”

Section: Introductionmentioning

confidence: 99%

Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning

Huang

Shao

Frangi

2018

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Multi-modality medical imaging is increasingly used for comprehensive assessment of complex diseases in either diagnostic examinations or as part of medical research trials. Different imaging modalities provide complementary information about living tissues. However, multi-modal examinations are not always possible due to adversary factors, such as patient discomfort, increased cost, prolonged scanning time, and scanner unavailability. In additionally, in large imaging studies, incomplete records are not uncommon owing to image artifacts, data corruption or data loss, which compromise the potential of multi-modal acquisitions. In this paper, we propose a weakly coupled and geometry co-regularized joint dictionary learning method to address the problem of cross-modality synthesis while considering the fact that collecting the large amounts of training data is often impractical. Our learning stage requires only a few registered multi-modality image pairs as training data. To employ both paired images and a large set of unpaired data, a cross-modality image matching criterion is proposed. Then, we propose a unified model by integrating such a criterion into the joint dictionary learning and the observed common feature space for associating cross-modality data for the purpose of synthesis. Furthermore, two regularization terms are added to construct robust sparse representations. Our experimental results demonstrate superior performance of the proposed model over state-of-the-art methods.

show abstract

Extended Modality Propagation: Image Synthesis of Pathological Cases

Cited by 41 publications

References 48 publications

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Deep‐learned short tau inversion recovery imaging using multi‐contrast MR images

Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning

Contact Info

Product

Resources

About