MR-contrast-aware image-to-image translations with generative adversarial networks

Abstract

Purpose A magnetic resonance imaging (MRI) exam typically consists of several sequences that yield different image contrasts. Each sequence is parameterized through multiple acquisition parameters that influence image contrast, signal-to-noise ratio, acquisition time, and/or resolution. Depending on the clinical indication, different contrasts are required by the radiologist to make a diagnosis. As MR sequence acquisition is time consuming and acquired images may be corrupted due to motion, a method to synthesize MR images with adjustable contrast properties is required. Methods Therefore, we trained an image-to-image generative adversarial network conditioned on the MR acquisition parameters repetition time and echo time. Our approach is motivated by style transfer networks, whereas the “style” for an image is explicitly given in our case, as it is determined by the MR acquisition parameters our network is conditioned on. Results This enables us to synthesize MR images with adjustable image contrast. We evaluated our approach on the fastMRI dataset, a large set of publicly available MR knee images, and show that our method outperforms a benchmark pix2pix approach in the translation of non-fat-saturated MR images to fat-saturated images. Our approach yields a peak signal-to-noise ratio and structural similarity of 24.48 and 0.66, surpassing the pix2pix benchmark model significantly. Conclusion Our model is the first that enables fine-tuned contrast synthesis, which can be used to synthesize missing MR-contrasts or as a data augmentation technique for AI training in MRI. It can also be used as basis for other image-to-image translation tasks within medical imaging, e.g., to enhance intermodality translation (MRI → CT) or 7 T image synthesis from 3 T MR images.

Synthesizing pseudo-T2w images to recapture missing data in neonatal neuroimaging with applications in rs-fMRI

“…Deep learning continues to allure researchers with promises of an all-powerful model capable of generating multi-contrast images with tunable image acquisition parameters ( Denck et al., 2021 ). While recent developments in the form of GANs have enabled training of such models without the requirement of paired data, sizable datasets with a wide variety of image types are still required.…”

SMU-Net: Style matching U-Net for brain tumor segmentation with missing modalities

“…The effectiveness of neural style transfer was also utilized in deep learning networks that process MR images. The authors of (Denck et al, 2021), for example, assert that contrast differentiation across MRI modalities would result in a style variation, therefor, they presented an image-to-image generative adversarial network capable of synthesizing MR pictures with configurable contrast via style transfer. The network presented by the authors of (Tomar et al, 2022) also employ a style encoder to group images with roughly similar styles together.…”

One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputation

“…In the recent years, many CNN-based approaches have demonstrated state-of-the-art performance for MR contrast synthesis [16]- [27]. Generative adversarial networks were introduced to enable the synthesis of realistic images across various domains [20], [23], [24]. While most of the literature focused on one-to-one synthesis, several studies [25]- [27] considered the many-to-one synthesis problem, where the algorithm takes multiple contrasts as input and generates one missing contrast.…”