Effect of the Pixel Interpolation Method for Downsampling Medical Images on Deep Learning Accuracy

Hirahara, Daisuke; Takaya, Eichi; Kadowaki, Mizuki; Kobayashi, Yasuyuki; Ueda, Takuya

doi:10.4236/jcc.2021.911010

Cited by 12 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, when LR images are generated by bicubic downsampling,high frequency components remaining after interpolation can affect the training of the network. 23,24 In natural RGB (red, green, and blue) images, previous studies have tried to generate more realistic LR training data by addressing the downsampling limitations. RUNet 25 was trained with LR images that were bicubically downsampled images and then blurred with a random Gaussian filter.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning in computed tomography super resolution using multi‐modality data training

Fok,

Fieselmann,

Herbst

et al. 2023

Medical Physics

View full text Add to dashboard Cite

BackgroundOne of the limitations in leveraging the potential of artificial intelligence in X‐ray imaging is the limited availability of annotated training data. As X‐ray and CT shares similar imaging physics, one could achieve cross‐domain data sharing, so to generate labeled synthetic X‐ray images from annotated CT volumes as digitally reconstructed radiographs (DRRs). To account for the lower resolution of CT and the CT‐generated DRRs as compared to the real X‐ray images, we propose the use of super‐resolution (SR) techniques to enhance the CT resolution before DRR generation.PurposeAs spatial resolution can be defined by the modulation transfer function kernel in CT physics, we propose to train a SR network using paired low‐resolution (LR) and high‐resolution (HR) images by varying the kernel's shape and cutoff frequency. This is different to previous deep learning‐based SR techniques on RGB and medical images which focused on refining the sampling grid. Instead of generating LR images by bicubic interpolation, we aim to create realistic multi‐detector CT (MDCT) like LR images from HR cone‐beam CT (CBCT) scans.MethodsWe propose and evaluate the use of a SR U‐Net for the mapping between LR and HR CBCT image slices. We reconstructed paired LR and HR training volumes from the same CT scans with small in‐plane sampling grid size of . We used the residual U‐Net architecture to train two models. SRUN: trained with kernel‐based LR images, and SRUN: trained with bicubic downsampled data as baseline. Both models are trained on one CBCT dataset (n = 13 391). The performance of both models was then evaluated on unseen kernel‐based and interpolation‐based LR CBCT images (n = 10 950), and also on MDCT images (n = 1392).ResultsFive‐fold cross validation and ablation study were performed to find the optimal hyperparameters. Both SRUN and SRUN models show significant improvements (p‐value < 0.05) in mean absolute error (MAE), peak signal‐to‐noise ratio (PSNR) and structural similarity index measures (SSIMs) on unseen CBCT images. Also, the improvement percentages in MAE, PSNR, and SSIM by SRUN is larger than SRUN. For SRUN, MAE is reduced by 14%, and PSNR and SSIMs increased by 6 and 8%, respectively. To conclude, SRUN outperforms SRUN, which the former generates sharper images when tested with kernel‐based LR CBCT images as well as cross‐modality LR MDCT data.ConclusionsOur proposed method showed better performance than the baseline interpolation approach on unseen LR CBCT. We showed that the frequency behavior of the used data is important for learning the SR features. Additionally, we showed cross‐modality resolution improvements to LR MDCT images. Our approach is, therefore, a first and essential step in enabling realistic high spatial resolution CT‐generated DRRs for deep learning training.

show abstract

Section: Introductionmentioning

confidence: 99%

“…GANs were also used for generating SR images with identity and cycle consistency loss on CT, 19,20 cone‐beam CT (CBCT), 21 and chest X‐ray 22 images. However, when LR images are generated by bicubic downsampling, high frequency components remaining after interpolation can affect the training of the network 23,24 …”

Section: Introductionmentioning

confidence: 99%

Deep learning in computed tomography super resolution using multi‐modality data training

Fok,

Fieselmann,

Herbst

et al. 2023

Medical Physics

View full text Add to dashboard Cite

show abstract

“…However, the resources required to run DL applications are not trivial, being these models typically overparameterized [3], and therefore requiring a high amount of computational resources. We refer here to the size of the learned DL models, and not to the evergrowing size of the datasets to be processed [28], or the size of the single images, as these issues have been addressed in literature (see, e.g., [25] and the use of tiles/patches [5]). Just to state some examples, depending on their specific implementation, the space needed to store a CNN trained for image classification purposes, such as the previously mentioned DenseNet, ResNet, or AlexNet models, varies from tens to hundreds of megabytes; this requirement jumps up to several tens gigabytes if we consider modern Large Language Models (e.g., 11 billions of parameters for some variants of the T5 Text-To-Text Transformer Model [42], meaning around 44 terabytes of RAM when using 4 bytes per parameter).…”

Section: Introductionmentioning

confidence: 99%

Resource-Limited Automated Ki67 Index Estimation in Breast Cancer

Gliozzo,

Marinò,

Bonometti

et al. 2023

Proceedings of the 2023 10th International Conference on Bioinformatics Research and Applications

View full text Add to dashboard Cite

The prediction of tumor progression and chemotherapy response has been recently tackled exploiting Tumor Infiltrating Lymphocytes (TILs) and the nuclear protein Ki67 as prognostic factors. Recently, deep neural networks (DNNs) have been shown to achieve top results in estimating Ki67 expression and simultaneous determination of intratumoral TILs score in breast cancer cells. However, in the last ten years the extraordinary progress induced by deep models proliferated at least as much as their resource demand. The exorbitant computational costs required to query (and in some cases also to store) a deep model represent a strong limitation in resource-limited contexts, like that of IoT-based applications to support healthcare personnel. To this end, we propose a resource consumption-aware DNN for the effective estimate of the percentage of Ki67-positive cells in breast cancer screenings. Our approach reduced up to 75% and 89% the usage of memory and disk space respectively, up to 1.5× the energy consumption, and preserved or improved the overall accuracy of a benchmark state-of-the-art solution. Encouraged by such positive results, we developed and structured the adopted framework so as to allow its general purpose usage, along with a public software repository to support its usage.

show abstract

“…In medical image AI, the impact of image resolution on accuracy has been demonstrated to be significant 5) . The impact of traditional image interpolation methods on the accuracy of DL models has already been studied 6) . However, recent DL-based super-resolution methods have not been studied.…”

Section: Introductionmentioning

confidence: 99%

Effect of Super Resolution on Low-Resolution MRI Segmentation

Takaya

Haraoka

Takahashi

et al. 2022

J St. Marianna Univ

Self Cite

View full text Add to dashboard Cite

Recently, numerous attempts have been devoted to applying deep-learning-based super resolution to medical images. However, discussions on its usefulness have been limited to the use of indices such as peak signalto-noise ratio (PSNR) and structural similarity (SSIM), and the significance of its application has not been widely discussed. This study aimed to compare several deep learning (DL) -based super-resolution methods using publicly available brain magnetic resonance imaging datasets. The impact of training the segmentation models on super-resolved images was also investigated. The results demonstrated the superiority of the DL-based model over traditional image interpolation methods and the limitations of its application in medical imaging. Additionally, the results indicated that PSNR and SSIM might not always be suitable as evaluation indices.

show abstract

Effect of the Pixel Interpolation Method for Downsampling Medical Images on Deep Learning Accuracy

Cited by 12 publications

References 16 publications

Deep learning in computed tomography super resolution using multi‐modality data training

Deep learning in computed tomography super resolution using multi‐modality data training

Resource-Limited Automated Ki67 Index Estimation in Breast Cancer

Effect of Super Resolution on Low-Resolution MRI Segmentation

Contact Info

Product

Resources

About