Simultaneous single- and multi-contrast super-resolution for brain MRI images based on a convolutional neural network

Zeng, Kun; Zheng, Hong; Cai, Congbo; Yang, Yu; Zhang, Kaihua; Chen, Zhong

doi:10.1016/j.compbiomed.2018.06.010

Cited by 120 publications

(119 citation statements)

References 25 publications

Supporting

Mentioning

117

Contrasting

Order By: Relevance

“…There are generally three methods to achieve image SR in MRI: (a) interpolation‐based, (b) reconstruction‐based, and (c) machine learning‐based . Interpolation‐based techniques assume that points/regions in an LR image can be expanded into corresponding points/regions in the SR reconstruction using polynomial or interpolation functions with some smoothness priors, which is not valid in inhomogeneous regions .…”

Section: Introductionmentioning

confidence: 99%

“…In contrast to the former approaches, the DL‐based method does not require the construction of a transformation model, but instead learns the direct mapping based on information from previously scanned datasets. Among these approaches, the convolutional neural network (CNN) is popular on account of its simple network structure and high accuracy …”

Section: Introductionmentioning

confidence: 99%

“…Although HR MRIs from patients previously scanned in the clinic are abundant, the corresponding LR images are not available. Current approaches in this case include preparing the training dataset from abundant HR images utilizing a simple translation model such as bicubic interpolation or k‐space down‐sampling . However, these approaches do not reflect real LR scans that are directly generated from MRI scanners, as properties of the signal (e.g., sampling bandwidth or noise) are often not feasible to model with a simple down‐sampling method.…”

Section: Introductionmentioning

confidence: 99%

“…2,18,[21][22][23][24][25] There are generally three methods to achieve image SR in MRI: (a) interpolation-based 9,26,27 , (b) reconstructionbased [28][29][30] , and (c) machine learning-based. [31][32][33][34][35] Interpolation-based techniques assume that points/regions in an LR image can be expanded into corresponding points/regions in the SR reconstruction using polynomial or interpolation functions with some smoothness priors, 9,26,27 which is not valid in inhomogeneous regions. 25 Moreover, the actual LR sampled points represent a nonideal sampling where the sampled points represent the intermediate value of the underlying HR points that exist within the LR points.…”

Section: Introductionmentioning

confidence: 99%

“…Among these approaches, the convolutional neural network (CNN) is popular on account of its simple network structure and high accuracy. [35][36][37][38] However, conventional DL-based methods have three major limitations. First, since MRIs have relatively lower image quality and more complex tissue structures than natural images, it is more challenging to restore high-frequency information.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

MRI super‐resolution reconstruction for MRI‐guided adaptive radiotherapy using cascaded deep learning: In the presence of limited training data and unknown translation model

et al. 2019

View full text Add to dashboard Cite

Purpose Deep learning (DL)‐based super‐resolution (SR) reconstruction for magnetic resonance imaging (MRI) has recently been receiving attention due to the significant improvement in spatial resolution compared to conventional SR techniques. Challenges hindering the widespread implementation of these approaches remain, however. Low‐resolution (LR) MRIs captured in the clinic exhibit complex tissue structures obfuscated by noise that are difficult for a simple DL framework to handle. Moreover, training a robust network for a SR task requires abundant, perfectly matched pairs of LR and high‐resolution (HR) images that are often unavailable or difficult to collect. The purpose of this study is to develop a novel SR technique for MRI based on the concept of cascaded DL that allows for the reconstruction of high‐quality SR images in the presence of insufficient training data, an unknown translation model, and noise. Methods The proposed framework, based on the concept named cascaded deep learning, consists of three components: (a) a denoising autoencoder (DAE) trained using clinical LR noisy MRI scans that have been processed with a nonlocal means filter that generates denoised LR data; (b) a down‐sampling network (DSN) trained with a small amount of paired LR/HR data from volunteers that allows for the generation of perfectly paired LR/HR data for the training of a generative model; and (c) the proposed SR generative model (p‐SRG) trained with data generated by the DSN that maps from LR inputs to HR outputs. After training, LR clinical images may be fed through the DAE and p‐SRG to yield SR reconstructions of the LR input. The application of this framework was explored in two settings: 3D breath‐hold MRI axial SR reconstruction from LR axial scans (<3 sec/vol) and in the enhancement of the spatial resolution of LR 4D‐MRI acquisitions (0.5 sec/vol). Results The DSN produces LR scans from HR inputs with a higher fidelity to true, LR clinical scans compared to conventional k‐space down‐sampling methods based on the metrics of root mean square error (RMSE) and structural similarity index (SSIM). Furthermore, HR outputs generated by the p‐SRG exhibit improved scores in the peak signal‐to‐noise ratio, normalized RMSE, SSIM, and in the blind/reference‐less image spatial quality evaluator assessment compared to conventional approaches to MRI SR. Conclusions The robust, SR reconstruction method for MRI based on the novel cascaded deep learning framework is an end‐to‐end method for producing detail‐preserving SR reconstructions from noisy, LR clinical MRI scans. Fourfold enhancements in spatial resolution facilitate target delineation and motion management during radiation therapy, enabling precise MRI‐guided radiation therapy with 3D LR breath‐hold MRI and 4D‐MRI in a clinically feasible time frame.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

MRI super‐resolution reconstruction for MRI‐guided adaptive radiotherapy using cascaded deep learning: In the presence of limited training data and unknown translation model

et al. 2019

View full text Add to dashboard Cite

show abstract

Accelerated Musculoskeletal Magnetic Resonance Imaging

Yoon,

Gold,

Chaudhari

2023

Magnetic Resonance Imaging

View full text Add to dashboard Cite

With a substantial growth in the use of musculoskeletal MRI, there has been a growing need to improve MRI workflow, and faster imaging has been suggested as one of the solutions for a more efficient examination process. Consequently, there have been considerable advances in accelerated MRI scanning methods. This article aims to review the basic principles and applications of accelerated musculoskeletal MRI techniques including widely used conventional acceleration methods, more advanced deep learning‐based techniques, and new approaches to reduce scan time. Specifically, conventional accelerated MRI techniques, including parallel imaging, compressed sensing, and simultaneous multislice imaging, and deep learning‐based accelerated MRI techniques, including undersampled MR image reconstruction, super‐resolution imaging, artifact correction, and generation of unacquired contrast images, are discussed. Finally, new approaches to reduce scan time, including synthetic MRI, novel sequences, and new coil setups and designs, are also reviewed. We believe that a deep understanding of these fast MRI techniques and proper use of combined acceleration methods will synergistically improve scan time and MRI workflow in daily practice.Evidence Level3Technical EfficacyStage 1

show abstract

On the proper use of structural similarity for the robust evaluation of medical image synthesis models

2022

View full text Add to dashboard Cite

To propose good practices for using the structural similarity metric (SSIM) and reporting its value. SSIM is one of the most popular image quality metrics in use in the medical image synthesis community because of its alleged superiority over voxel-by-voxel measurements like the average error or the peak signal noise ratio (PSNR). It has seen massive adoption since its introduction, but its limitations are often overlooked. Notably, SSIM is designed to work on a strictly positive intensity scale, which is generally not the case in medical imaging. Common intensity scales such as the Houndsfield units (HU) contain negative numbers, and they can also be introduced by image normalization techniques such as the z-normalization. Methods: We created a series of experiments to quantify the impact of negative values in the SSIM computation. Specifically, we trained a three-dimensional (3D) U-Net to synthesize T2-weighted MRI from T1-weighted MRI using the BRATS 2018 dataset. SSIM was computed on the synthetic images with a shifted dynamic range. Next, to evaluate the suitability of SSIM as a loss function on images with negative values, it was used as a loss function to synthesize znormalized images. Finally, the difference between two-dimensional (2D) SSIM and 3D SSIM was investigated using multiple 2D U-Nets trained on different planes of the images. Results:The impact of the misuse of the SSIM was quantified; it was established that it introduces a large downward bias in the computed SSIM. It also introduces a small random error that can change the relative ranking of models. The exact values for this bias and error depend on the quality and the intensity histogram of the synthetic images. Although small, the reported error is significant considering the small SSIM difference between state-of -the-art models. It was shown therefore that SSIM cannot be used as a loss function when images contain negative values due to major errors in the gradient calculation, resulting in under-performing models. 2D SSIM was also found to be overestimated in 2D image synthesis models when computed along the plane of synthesis, due to the discontinuities between slices that is typical of 2D synthesis methods. Conclusion: Various types of misuse of the SSIM were identified, and their impact was quantified. Based on the findings, this paper proposes good practices when using SSIM, such as reporting the average over the volume of the image containing tissue and appropriately defining the dynamic range.

show abstract

Simultaneous single- and multi-contrast super-resolution for brain MRI images based on a convolutional neural network

Cited by 120 publications

References 25 publications

MRI super‐resolution reconstruction for MRI‐guided adaptive radiotherapy using cascaded deep learning: In the presence of limited training data and unknown translation model

MRI super‐resolution reconstruction for MRI‐guided adaptive radiotherapy using cascaded deep learning: In the presence of limited training data and unknown translation model

Accelerated Musculoskeletal Magnetic Resonance Imaging

On the proper use of structural similarity for the robust evaluation of medical image synthesis models

Contact Info

Product

Resources

About