Inpainting the metal artifact region in MRI images by using generative adversarial networks with gated convolution

Abstract

Magnetic resonance imaging (MRI) plays an important role in clinical diagnosis, but it is susceptible to metal artifacts. The generative adversarial network GatedConv with gated convolution (GC) and contextual attention (CA) was used to inpaint the metal artifact region in MRI images. Methods: MRI images containing or near the teeth of 70 patients were collected, and the scanning sequence was a T1-weighted high-resolution isotropic volume examination sequence. A total of 10 000 slices were obtained after data enhancement, of which 8000 slices were used for training. MRI images were normalized to [−1,1]. Based on the randomly generated mask, U-Net, pix2pix, PConv with partial convolution, and GatedConv were used to inpaint the artifact region of MRI images. The mean absolute error (MAE) and peak signal-to-noise ratio (PSNR) for the mask were used to compare the results of these methods. The inpainting effect on the test dataset using dental masks was also evaluated. Besides, the artifact area of clinical MRI images was inpainted based on the mask sketched by physicians. Finally, the earring artifacts and artifacts caused by abnormal signal foci were inpainted to verify the generalization of the models. Results: GatedConv could directly and effectively inpaint the incomplete MRI images generated by masks in the image domain. For the results of U-Net, pix2pix, PConv, and GatedConv, the masked MAEs were 0.1638, 0.1812, 0.1688, and 0.1596, respectively, and the masked PSNRs were 18.2136, 17.5692, 18.2258, and 18.3035 dB, respectively. Using dental masks, the results of U-Net, pix2pix, and PConv differed more from the real images in terms of alveolar shape and surrounding tissue compared with GatedConv. GatedConv could inpaint the metal artifact region in clinical MRI images more effectively than the other models, but the increase in the mask area could reduce the inpainting effect. Inpainted MRI images by GatedConv and CT images with metal artifact reduction coincided with alveolar and tissue structure, and GatedConv could successfully inpaint artifacts caused by abnormal signal foci, whereas the other models failed. The ablation study demonstrated that GC and CA increased the reliability of the inpainting performance of GatedConv. Conclusion: MRI images are affected by metal, and signal void areas appear near metal. GatedConv can inpaint the MRI metal artifact region in the image domain directly and effectively and improve image quality. Medical image inpainting by GatedConv has potential value for tasks,such as positron emission tomography (PET) attenuation correction in PET/MRI and adaptive radiotherapy of synthetic CT based on MRI.Kai Xie and Liugang Gao equal contribution.

Artificial Intelligence–Driven Single-Shot PET Image Artifact Detection and Disentanglement

“…5,30 The presence of metallic objects and prostheses, pacemakers, and oral/IV contrast agents, catheters, coiling, spine rods, and calcified lymph nodes result in photon starvation, beam hardening, and streak artifacts (because of high photon absorption) in CT and void signals in MRI. [31][32][33] The aforementioned high-density materials do not disturb or decrease genuine PET image signals. [31][32][33] However, they deteriorate CT image quality in dense regions and adjacent organs that propagate to PET images.…”

“…[31][32][33] The aforementioned high-density materials do not disturb or decrease genuine PET image signals. [31][32][33] However, they deteriorate CT image quality in dense regions and adjacent organs that propagate to PET images. Metal artifacts result in CT signal skewing, which overestimates or underestimates corresponding tissue Hounsfield units (HUs), which in turns tends to overcorrect/undercorrect PET data.…”

“…Metal artifacts result in CT signal skewing, which overestimates or underestimates corresponding tissue Hounsfield units (HUs), which in turns tends to overcorrect/undercorrect PET data. [31][32][33] In PET/CT scanners, the presence of the aforementioned highly dense objects could mimic intense radiotracer uptake overestimating the SUV by up to 20%, 34 which can potentially be interpreted as an abnormality or malignant lesion, thus increasing the false-positive rate. Metallic objects and their corresponding artifacts could be easily detected on CT images.…”

“…35 Nonetheless, such artifacts continue to challenge PET/CT because of the wide variety of material compositions, locations, and sizes of metallic objects. [31][32][33] Conventional metal artifact correction often exhibits suboptimal performance (significantly over/ underestimating CT HUs) and might introduce new artifacts. [31][32][33] There is a need for effective and feasible techniques to capture and compensate for such artifacts; otherwise, they can degrade PET images with time, cost, radiation dose, and patient comfort implications.…”

“…[31][32][33] Conventional metal artifact correction often exhibits suboptimal performance (significantly over/ underestimating CT HUs) and might introduce new artifacts. [31][32][33] There is a need for effective and feasible techniques to capture and compensate for such artifacts; otherwise, they can degrade PET images with time, cost, radiation dose, and patient comfort implications. 8 Meanwhile, some artifacts are inevitable and cannot be corrected by repeating image acquisition.…”