Noise2Recon: A Semi-Supervised Framework for Joint MRI Reconstruction and Denoising

Desai, Arjun; Ozturkler, Batu; Sandino, Christopher M.; Vasanawala, Shreyas S.; Hargreaves, Brian A.; Ré, Christopher; Pauly, John M.; Chaudhari, Akshay

doi:10.48550/arxiv.2110.00075

Cited by 4 publications

(6 citation statements)

References 29 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3) Future Work: GLEAM could be extended to highdimensional imaging scenarios such as DCE, 4D-flow, or 3D non-cartesian imaging, and to other learning settings such as semi-supervised [38], [39] or self-supervised [40]…”

Section: Discussionmentioning

confidence: 99%

GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction

Ozturkler¹,

Sahiner²,

Ergen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Unrolled neural networks have recently achieved state-of-the-art accelerated MRI reconstruction. These networks unroll iterative optimization algorithms by alternating between physics-based consistency and neural-network based regularization. However, they require several iterations of a large neural network to handle high-dimensional imaging tasks such as 3D MRI. This limits traditional training algorithms based on backpropagation due to prohibitively large memory and compute requirements for calculating gradients and storing intermediate activations. To address this challenge, we propose Greedy LEarning for Accelerated MRI (GLEAM) reconstruction, an efficient training strategy for high-dimensional imaging settings. GLEAM splits the end-to-end network into decoupled network modules. Each module is optimized in a greedy manner with decoupled gradient updates, reducing the memory footprint during training. We show that the decoupled gradient updates can be performed in parallel on multiple graphical processing units (GPUs) to further reduce training time. We present experiments with 2D and 3D datasets including multi-coil knee, brain, and dynamic cardiac cine MRI. We observe that: i) GLEAM generalizes as well as state-of-the-art memory-efficient baselines such as gradient checkpointing and invertible networks with the same memory footprint, but with 1.3x faster training; ii) for the same memory footprint, GLEAM yields 1.1dB PSNR gain in 2D and 1.8 dB in 3D over end-to-end baselines.

show abstract

Section: Discussionmentioning

confidence: 99%

GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction

Ozturkler¹,

Sahiner²,

Ergen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Despite their label efficiency, these techniques still underperform supervised methods and are also sensitive to distribution shift. Recently, a family of semi-supervised reconstruction methods demonstrated label efficiency and robustness to physics-driven perturbations, such as changes in signalto-noise ratio or patient motion [19,18]. However, these methods require large amounts of unlabeled data, which can be difficult to curate in few-shot settings.…”

Section: A Extended Related Workmentioning

confidence: 99%

“…We denote the class of all matrices M expressible in this form by M ((b1,b2,b3),(n1,n2,n3)) . Observe that when b 19.…”

Section: Definition C19 (Rectangular Monarch Matrixmentioning

confidence: 99%

Monarch: Expressive Structured Matrices for Efficient and Accurate Training

Dao¹,

Chen²,

Sohoni³

et al. 2022

Preprint

View full text Add to dashboard Cite

Large neural networks excel in many domains, but they are expensive to train and fine-tune. A popular approach to reduce their compute/memory requirements is to replace dense weight matrices with structured ones (e.g., sparse, low-rank, Fourier transform). These methods have not seen widespread adoption (1) in end-to-end training due to unfavorable efficiency-quality tradeoffs, and (2) in denseto-sparse fine-tuning due to lack of tractable algorithms to approximate a given dense weight matrix. To address these issues, we propose a class of matrices (Monarch) that is hardware-efficient (they are parameterized as products of two block-diagonal matrices for better hardware utilization) and expressive (they can represent many commonly used transforms). Surprisingly, the problem of approximating a dense weight matrix with a Monarch matrix, though nonconvex, has an analytical optimal solution. These properties of Monarch matrices unlock new ways to train and fine-tune sparse and dense models. We empirically validate that Monarch can achieve favorable accuracy-efficiency tradeoffs in several end-to-end sparse training applications: speeding up ViT and GPT-2 training on ImageNet classification and Wikitext-103 language modeling by 2× with comparable model quality, and reducing the error on PDE solving and MRI reconstruction tasks by 40%. In sparse-to-dense training, with a simple technique called "reverse sparsification," Monarch matrices serve as a useful intermediate representation to speed up GPT-2 pretraining on OpenWebText by 2× without quality drop. The same technique brings 23% faster BERT pretraining than even the very optimized implementation from Nvidia that set the MLPerf 1.1 record. In dense-to-sparse fine-tuning, as a proof-of-concept, our Monarch approximation algorithm speeds up BERT fine-tuning on GLUE by 1.7× with comparable accuracy.

show abstract

“…Fabian et al [12] proposed MRAugment, an image-based data augmentation pipeline. Desai et al proposed a semi-supervised consistency framework for joint reconstruction and denoising [11], and later extended the denoising objective to a generalized data augmentation pipeline that enables composing a broader family of physics-driven acquisition-based augmentations and image-based augmentations [10].…”

Section: Related Workmentioning

confidence: 99%

“…However, these approaches still depend on fully-sampled scans as ground truth data which is either costly or not possible to acquire in many clinical medical imaging applications; hence, reducing dependence on data, referred to as data-efficiency, is desirable. Towards the goal of improving data-efficiency, besides the prior proposals which leverage prospectively undersampled (unsupervised) data that currently lag in reconstruction performance [8,9,13], recent work has proposed designing data augmentation pipelines tailored to accelerated MRI reconstruction with appropriate image-based [12] or acquisition-based, physics-driven transformations [11,10]. Although helpful with data efficiency and robustness to certain distribution drifts, these approaches do not guarantee that the final reconstruction model satisfies the desired symmetries, introduced through the data augmentation transformations, at train or inference time -which may increase the existing concerns among clinicians around using data-driven techniques.…”

Section: Introductionmentioning

confidence: 99%

Scale-Equivariant Unrolled Neural Networks for Data-Efficient Accelerated MRI Reconstruction

Gunel¹,

Sahiner²,

Desai³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Unrolled neural networks have enabled state-of-the-art reconstruction performance and fast inference times for the accelerated magnetic resonance imaging (MRI) reconstruction task. However, these approaches depend on fully-sampled scans as ground truth data which is either costly or not possible to acquire in many clinical medical imaging applications; hence, reducing dependence on data is desirable. In this work, we propose modeling the proximal operators of unrolled neural networks with scale-equivariant convolutional neural networks in order to improve the data-efficiency and robustness to drifts in scale of the images that might stem from the variability of patient anatomies or change in field-of-view across different MRI scanners. Our approach demonstrates strong improvements over the state-of-the-art unrolled neural networks under the same memory constraints both with and without data augmentations on both in-distribution and out-of-distribution scaled images without significantly increasing the train or inference time.

show abstract

Noise2Recon: A Semi-Supervised Framework for Joint MRI Reconstruction and Denoising

Cited by 4 publications

References 29 publications

GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction

GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction

Monarch: Expressive Structured Matrices for Efficient and Accurate Training

Scale-Equivariant Unrolled Neural Networks for Data-Efficient Accelerated MRI Reconstruction

Contact Info

Product

Resources

About