MRI Restoration Using Edge-Guided Adversarial Learning

Chai, Yaqiong; Xu, Botian; Zhang, Kangning; Leporé, Natasha; Wood, John C.

doi:10.1109/access.2020.2992204

Cited by 21 publications

(12 citation statements)

References 60 publications

(75 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these methods do not use the inherent structure of information in the medical images, resulting in blurry images and often lacking detail. The authors in [7] proposed a method using structural information which is represented by the edges of the image. The network decouples image repair into two separate stages: edge connection and contrast completion.…”

Section: Learning-based Approachmentioning

confidence: 99%

“…These techniques still suffer from incomplete restorations, such as blurred boundaries and loss of the organ structures inside the deformed part. To overcome such problems of restoration failures, structural information has been exploited, where the information of edges is the main tool for implementing the learning process [4][5][6][7][8]. EdgeConnect [4] is a two-stage adversarial model that comprises an edge generator followed by an image completion network.…”

Section: Introductionmentioning

confidence: 99%

“…The foreground contours are predicted first and then the inpainting is performed using the predicted contours as a guidance. The Edge-Guided GAN [7] method is an edge-guided generative adversarial network to restore brain MRI images which have distortion or missing parts. A multi-task learning framework with auxiliary tasks of edge finding and gradient map prediction is used to incorporate the knowledge of the image structure to assist inpainting [8].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness

et al. 2021

View full text Add to dashboard Cite

Distorted medical images can significantly hamper medical diagnosis, notably in the analysis of Computer Tomography (CT) images and organ segmentation specifics. Therefore, improving diagnostic imagery accuracy and reconstructing damaged portions are important for medical diagnosis. Recently, these issues have been studied extensively in the field of medical image inpainting. Inpainting techniques are emerging in medical image analysis since local deformations in medical modalities are common because of various factors such as metallic implants, foreign objects or specular reflections during the image captures. The completion of such missing or distorted regions is important for the enhancement of post-processing tasks such as segmentation or classification. In this paper, a novel framework for medical image inpainting is presented by using a multi-task learning model for CT images targeting the learning of the shape and structure of the organs of interest. This novelty has been accomplished through simultaneous training for the prediction of edges and organ boundaries with the image inpainting, while state-of-the-art methods still focus only on the inpainting area without considering the global structure of the target organ. Therefore, our model reproduces medical images with sharp contours and exact organ locations. Consequently, our technique generates more realistic and believable images compared to other approaches. Additionally, in quantitative evaluation, the proposed method achieved the best results in the literature so far, which include a PSNR value of 43.44 dB and SSIM of 0.9818 for the square-shaped regions; a PSNR value of 38.06 dB and SSIM of 0.9746 for the arbitrary-shaped regions. The proposed model generates the sharp and clear images for inpainting by learning the detailed structure of organs. Our method was able to show how promising the method is when applying it in medical image analysis, where the completion of missing or distorted regions is still a challenging task.

show abstract

Section: Learning-based Approachmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness

et al. 2021

View full text Add to dashboard Cite

show abstract

“…we observed a higer F1 score than nsgan loss. The edge guide method useful not only for image outpainting, but also for image inpainting [22], super-resolution [23], and various other fields [6,16]. Therefore, it is believed that the hinge loss in other fields as well as binary data such as edge maps will help improve the performance.…”

Section: Hinge Lossmentioning

confidence: 99%

Painting Outside as Inside: Edge Guided Image Outpainting via Bidirectional Rearrangement with Progressive Step Learning

Kim

Yun

Kang

et al. 2020

Preprint

View full text Add to dashboard Cite

Image outpainting is a very intriguing problem as the outside of a given image can be continuously filled by considering as the context of the image. This task has two main challenges. The first is to maintain the spatial consistency in contents of generated regions and the original input. The second is to generate a high-quality large image with a small amount of adjacent information. Conventional image outpainting methods generate inconsistent, blurry, and repeated pixels. To alleviate the difficulty of an outpainting problem, we propose a novel image outpainting method using bidirectional boundary region rearrangement. We rearrange the image to benefit from the image inpainting task by reflecting more directional information. The bidirectional boundary region rearrangement enables the generation of the missing region using bidirectional information similar to that of the image inpainting task, thereby generating the higher quality than the conventional methods using unidirectional information. Moreover, we use the edge map generator that considers images as original input with structural information and hallucinates the edges of unknown regions to generate the image. Our proposed method is compared with other state-of-the-art outpainting and inpainting methods both qualitatively and quantitatively. We further compared and evaluated them using BRISQUE, one of the No-Reference image quality assessment (IQA) metrics, to evaluate the naturalness of the output. The experimental results demonstrate that our method outperforms other methods and generates new images with 360°panoramic characteristics.Figure 1. Main concept of the proposed method. We rearrange the outer area of the image into its inner area. This enables us to consider bidirectional information, and it can generate a high-quality natural image that is superior to images produced by conventional methods.

show abstract

“…Ea-GANs contain a generator-induced gEa-GAN, and a discriminator-induced dEa-GAN, for enriching the reconstruction images with more details. Chai et al [ 55 ] designed an edge-guided GAN (EG-GAN), to restore brain MRI images which decoupled reconstruction into edge connection and contrast completion. Li et al [ 56 ], proposed a dual-discriminator GAN, i.e., EDDGAN, of which one discriminator was used for holistic image reconstruction and the other one was for edge information preservation.…”

Section: Introductionmentioning

confidence: 99%

Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information

Huang

Ding

et al. 2022

Appl Intell

View full text Add to dashboard Cite

In clinical medicine, magnetic resonance imaging (MRI) is one of the most important tools for diagnosis, triage, prognosis, and treatment planning. However, MRI suffers from an inherent slow data acquisition process because data is collected sequentially in k-space. In recent years, most MRI reconstruction methods proposed in the literature focus on holistic image reconstruction rather than enhancing the edge information. This work steps aside this general trend by elaborating on the enhancement of edge information. Specifically, we introduce a novel parallel imaging coupled dual discriminator generative adversarial network (PIDD-GAN) for fast multi-channel MRI reconstruction by incorporating multi-view information. The dual discriminator design aims to improve the edge information in MRI reconstruction. One discriminator is used for holistic image reconstruction, whereas the other one is responsible for enhancing edge information. An improved U-Net with local and global residual learning is proposed for the generator. Frequency channel attention blocks (FCA Blocks) are embedded in the generator for incorporating attention mechanisms. Content loss is introduced to train the generator for better reconstruction quality. We performed comprehensive experiments on Calgary-Campinas public brain MR dataset and compared our method with state-of-the-art MRI reconstruction methods. Ablation studies of residual learning were conducted on the MICCAI13 dataset to validate the proposed modules. Results show that our PIDD-GAN provides high-quality reconstructed MR images, with well-preserved edge information. The time of single-image reconstruction is below 5ms, which meets the demand of faster processing.

show abstract

MRI Restoration Using Edge-Guided Adversarial Learning

Cited by 21 publications

References 60 publications

Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness

Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness

Painting Outside as Inside: Edge Guided Image Outpainting via Bidirectional Rearrangement with Progressive Step Learning

Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information

Contact Info

Product

Resources

About