C-arm X-ray imaging is commonly applied in operating rooms for guiding orthopedic surgeries. Augmented Reality (AR) with C-arm X-ray images during surgery is an efficient way to facilitate procedures for surgeons. However, the accurate calibration process for surgical AR based on C-arm is essential and still challenging due to the limitations of C-arm imaging systems, such as instability of C-arm calibration parameters and the narrow field of view. We extend existing methods using a depth camera and propose a new calibration procedure consisting of calibration of the C-arm imaging system, and 3D/2D calibration of an RGB-D camera and C-arm system with a new method to achieve reliable data and promising accuracy and, at the same time, consistent with standard surgical protocols. For the calibration procedure, we apply bundle adjustment equations with a 3D designed Lego multi-modal phantom, in contrast to the previous methods in which planar calibration phantoms were applied. By using our method, the visualization of the X-ray image upon the 3D data was done, and the achieved mean overlay error was 1.03 mm. The evaluations showed that the proposed calibration procedure provided promising accuracy for AR surgeries and it improved the flexibility and robustness of existing C-arm calibration methods for surgical augmented reality (using C-arm and RGB-D sensor). Moreover, the results showed the efficiency of our method to compensate for the effects of the C-arm movement on calibration parameters. It was shown that the obtained overlay error was improved for the non-zero rotation movement of C-arm by using a virtual detector.Sensors 2020, 20, 36 2 of 16 as reported in [3][4][5], and (b) AR calibration for the integration of additional sensor such as the depth camera to the C-arm.In the following, related works for C-arm surgical AR applications, and the instability of C-arm calibration parameters are explored. Then, the contributions of this study are declared.
Research Overview for AR systems with C-armNavab et al.[1] proposed an AR system with video augmentation for C-arm devices in operating rooms by using mirrors. The captured X-ray images were registered with the video images by using a homography matrix. This AR system is limited to 2D guidance. For using 3D information, Habert et al. [6] applied a depth camera augmented on the C-arm by mirror construction, but this construction needed technical modifications to the C-arm device, which contradicted the minimal disruption aim [7]. It also limited the field of view and workspace of the surgeon [1,6].Habert et al. [8] and Habert [7] attached RGB-D sensors to the C-arm X-ray source and applied Zhang's method [9] with a multi-modal planar phantom in order to obtain the calibration parameters without mirrors. For calibration in the surgical AR navigation with C-arm, current studies, such as [10][11][12][13][14], have also used Zhang's method [9].In addition, there are some studies, such as [15,16], for registration of Computed Tomography (CT) data to intra-operative 3...