Swin Transformer for Fast MRI

Huang, Jiahao; Fang, Yingying; Wu, Yinzhe; Wu, Huanjun; Gao, Zhifan; Li, Yang; Ser, Javier Del; Xia, Jun; Yang, Guang

doi:10.48550/arxiv.2201.03230

Cited by 1 publication

(1 citation statement)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even though some work exists on Transformer-based architectures for supervised accelerated MRI reconstruction [Huang et al, 2022, Feng et al, 2021, to the best of our knowledge ours is the first work to demonstrate state-of-the-art results on large-scale MRI datasets such as the fastMRI dataset.…”

Section: Related Workmentioning

confidence: 97%

HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction

Fabian¹,

Soltanolkotabi²

2022

Preprint

View full text Add to dashboard Cite

In accelerated MRI reconstruction, the anatomy of a patient is recovered from a set of undersampled and noisy measurements. Deep learning approaches have been proven to be successful in solving this ill-posed inverse problem and are capable of producing very high quality reconstructions. However, current architectures heavily rely on convolutions, that are content-independent and have difficulties modeling long-range dependencies in images. Recently, Transformers, the workhorse of contemporary natural language processing, have emerged as powerful building blocks for a multitude of vision tasks. These models split input images into non-overlapping patches, embed the patches into lower-dimensional tokens and utilize a self-attention mechanism that does not suffer from the aforementioned weaknesses of convolutional architectures. However, Transformers incur extremely high compute and memory cost when 1) the input image resolution is high and 2) when the image needs to be split into a large number of patches to preserve fine detail information, both of which are typical in low-level vision problems such as MRI reconstruction, having a compounding effect. To tackle these challenges, we propose HUMUS-Net, a hybrid architecture that combines the beneficial implicit bias and efficiency of convolutions with the power of Transformer blocks in an unrolled and multi-scale network. HUMUS-Net extracts high-resolution features via convolutional blocks and refines low-resolution features via a novel Transformer-based multi-scale feature extractor. Features from both levels are then synthesized into a high-resolution output reconstruction. Our network establishes new state of the art on the largest publicly available MRI dataset, the fastMRI dataset. We further demonstrate the performance of HUMUS-Net on two other popular MRI datasets and perform fine-grained ablation studies to validate our design. We note that the current experimental results are preliminary, we will update this paper soon with our final results involving larger models.

show abstract

Section: Related Workmentioning

confidence: 97%