Projection-to-Projection Translation for Hybrid X-ray and Magnetic Resonance Imaging

Abstract

Hybrid X-ray and magnetic resonance (MR) imaging promises large potential in interventional medical imaging applications due to the broad variety of contrast of MRI combined with fast imaging of X-ray-based modalities. To fully utilize the potential of the vast amount of existing image enhancement techniques, the corresponding information from both modalities must be present in the same domain. For image-guided interventional procedures, X-ray fluoroscopy has proven to be the modality of choice. Synthesizing one modality from another in this case is an ill-posed problem due to ambiguous signal and overlapping structures in projective geometry. To take on these challenges, we present a learning-based solution to MR to X-ray projection-to-projection translation. We propose an image generator network that focuses on high representation capacity in higher resolution layers to allow for accurate synthesis of fine details in the projection images. Additionally, a weighting scheme in the loss computation that favors high-frequency structures is proposed to focus on the important details and contours in projection imaging. The proposed extensions prove valuable in generating X-ray projection images with natural appearance. Our approach achieves a deviation from the ground truth of only 6% and structural similarity measure of 0.913 ± 0.005. In particular the high frequency weighting assists in generating projection images with sharp appearance and reduces erroneously synthesized fine details.

Cephalogram Synthesis and Landmark Detection in Dental Cone-Beam CT Systems

“…Due to the severe perspective deformation in cone-beam projections, it is very challenging to restore such deformation with conventional methods. Recently, deep learning methods, particularly using generative adversarial networks (GANs) [20], have achieved promising results in image synthesis in various medical applications such as 3T MRI images to 7T MRI images [21], PET images to CT images [22], and MRI cone-beam projections to X-ray cone-beam projections [23]. However, to the best of our knowledge, such projection-to-cephalogram synthesis using GANs have not been investigated yet.…”

Learning Perspective Deformation in X-Ray Transmission Imaging

“…However, obtaining such 3D CBCT volumes brings additional dose exposure to patients, since hundreds of projections are acquired for 3D reconstruction. Another potential application is in hybrid magnetic resonance imaging (MRI) and X-ray imaging [14], [15]. Obtaining 3D MRI volumes to generate DRRs is time consuming, taking around 30 min for each scan.…”