Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers

Abstract

Supervised deep learning has swiftly become a workhorse for accelerated MRI in recent years, offering state-of-the-art performance in image reconstruction from undersampled acquisitions. Training deep supervised models requires large datasets of undersampled and fullysampled acquisitions typically from a matching set of subjects. Given scarce access to large medical datasets, this limitation has sparked interest in unsupervised methods that reduce reliance on fully-sampled ground-truth data. A common framework is based on the deep image prior, where network-driven regularization is enforced directly during inference on undersampled acquisitions. Yet, canonical convolutional architectures are suboptimal in capturing long-range relationships, and randomly initialized networks may hamper convergence. To address these limitations, here we introduce a novel unsupervised MRI reconstruction method based on zero-Shot Learned Adversarial TransformERs (SLATER). SLATER embodies a deep adversarial network with cross-attention transformer blocks to map noise and latent variables onto MR images. This unconditional network learns a high-quality MRI prior in a self-supervised encoding task. A zero-shot reconstruction is performed on undersampled test data, where inference is performed by optimizing network parameters, latent and noise variables to ensure maximal consistency to multi-coil MRI data. Comprehensive experiments on brain MRI datasets clearly demonstrate the superior performance of SLATER against state-of-the-art unsupervised methods.

Transformers in Medical Imaging: A Survey

“…We want to highlight one particular work that uses the Transformer-layer architecture to regularize the challenging problem of MRI image reconstruction from under-sampled measurements [38]. This work is inspired by the strong prior induced by the structure of untrained neural networks [46], [47].…”

“…Further, unlike deep learning-based approaches, they do not require large annotated medical imaging datasets for training. This reduced reliance on labeled datasets is crucial to the medical research community [34], (b) [35], (c) [36], (d) [37], (e) [38], (f) [39].…”

“…It can be seen that SLATTER outperforms all other approaches in term of quality of reconstruction. Results taken from [38].…”

“…One drawback of the aforementioned approaches is the requirement of a massive paired dataset of undersampled and corresponding fully sampled MRI acquisitions to train ViT models. To alleviate the data requirement issue, Korkmaz et al [38], [300] propose a zero-shot framework, SLATER, that leverages prior induced by randomly initialized neural networks [46], [47] for unsupervised MR image reconstruction. Specifically, during inference, SLATER inverts its transformer-based generative model via iterative optimization over-network weights to minimize the error between the network output and the under-sampled multi-coil MRI acquisitions while satisfying the MRI forward model constraints.…”

Class-Aware Generative Adversarial Transformers for Medical Image Segmentation

“…Moreover, using transformers has been shown to be more promising in computer vision (Dosovitskiy et al, 2020; for utilizing long-range dependencies than other, traditional CNN-based methods. In parallel, transformers with powerful global relation modeling abilities have become the standard starting point for training on a wide range of downstream medical imaging analysis tasks, such as image segmentation Cao et al, 2021;Wang et al, 2021b;Valanarasu et al, 2021;Xie et al, 2021b), image synthesis (Kong et al, 2021;Ristea et al, 2021;Dalmaz et al, 2021), and image enhancement (Korkmaz et al, 2021;Luthra et al, 2021;Wang et al, 2021a).…”