MRI to C‐arm spine registration through Pseudo‐3D CycleGANs with differentiable histograms

Abstract

Purpose Image‐guided spine surgery increasingly relies on diagnostic MRI for device navigation, as it allows to visualize the nerves and soft tissues during screw insertion in the pedicle region, which is not possible with preoperative CT or cone beam CT. However, registration of MRI to C‐arm images remains difficult due to differences in visible tissue. Methods In this paper, we introduce a three‐dimensional/two‐dimensional (3D/2D) registration method of preoperative T2‐weighted MRI of the lumbar spine to C‐arm X‐ray using synthetic CT images. The registration work is based on a pseudo‐3D CycleGAN integrating a new cyclic loss function to ensure consistency in MRI and CT synthesis using differentiable histograms to match the multimodal distributions. The unified framework allows to improve bony tissue inference as opposed to regular 2D CycleGAN for image synthesis. A multiplanar digitally reconstructed radiograph (DRR) registration approach aligns the 3D and 2D images. Results Experiments performed on a public dataset of 18 pathological spines yielded a mean dice coefficient of 0.84 ± 0.015 on synthetic CTs. The DRR registration experiments, on the other hand, presented a target localization error of 2.1 ± 0.2mm. Conclusion Intensity distributions and voxel‐wise errors in Hounsfield units show encouraging results, illustrating the network's flexibility of producing qualitatively and quantitatively reasonable synthetic CT scans that can be used in a surgical 3D/2D registration framework. These promising results demonstrate the potential of the synthesis tool prior to integration in an image‐guidance system.

Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration

“…Common similarity measures are gradient- [37], intensity- [38], and feature-based [39] metrics. More recently, 2D-3D registration methods have evolved to include learning-based algorithms to address challenges in cross-modality registration [40] and feature extraction [41].…”

3D MRI of the Spine

“…65,66 To date, synthetic CT data obtained from MRI have not been used successfully with neuronavigational software and are still under investigation. 67 Thorough evaluation of synthetic MRI in spine applications may yield further time savings and expanded applications for both 2D and 3D protocols.…”