Stereoscopic augmented reality system for supervised training on minimal invasive surgery robots

Matu, Florin Octavian; Thøgersen, Mikkel; Galsgaard, Bo; Jensen, Martin Møller; Kraus, Martin

doi:10.1145/2617841.2620722

Cited by 14 publications

(8 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The camera pose is detected and estimated by the Camera Calibration toolbox. 1 The example of images captured by a NanEye camera is shown in Fig. 9(a)-9(b).…”

Section: B Experiments With Real Datamentioning

confidence: 99%

“…Robotic-assisted minimally invasive surgery (RMIS) uses tele-manipulation setups to control the surgical instruments thereby enhancing ergonomics for the surgeon and providing better instrument articulation. The tele-manipulation configuration also provides a suitable platform with potential to introduce computer assisted interventions (CAI) in the operation such as virtual fixtures or overlaying intra-and pre-operative imaging onto the video feed for enhanced visualisation of structural and functional anatomical information underneath the visible tissue surface [1].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hand-Eye Calibration With a Remote Centre of Motion

Pachtrachai

Vasconcelos

Dwyer

et al. 2019

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

In the eye-in-hand robot configuration, hand-eye calibration plays a vital role in completing the link between the robot and camera coordinate systems. Calibration algorithms are mature and provide accurate transformation estimations for an effective camera-robot link but rely on a sufficiently wide range of calibration data to avoid errors and degenerate configurations. This can be difficult in the context of keyhole surgical robots because they are mechanically constrained to move around a remote centre of motion (RCM) which is located at the trocar port. The trocar limits the range of feasible calibration poses that can be obtained and results in ill-conditioned hand-eye constraints. In this paper, we propose a new approach to deal with this problem by incorporating the RCM constraints into the hand-eye formulation. We show that this not only avoids illconditioned constraints but is also more accurate than classic hand-eye calibration with a free 6DoF motion, due to solving simpler equations that take advantage of the reduced DoF. We validate our method using simulation to test numerical stability and a physical implementation on an RCM constrained KUKA LBR iiwa 14 R820 equipped with a NanEye stereo camera.

show abstract

“…The camera pose is detected and estimated by the Camera Calibration toolbox. 1 The example of images captured by a NanEye camera is shown in Fig. 9(a)-9(b).…”

Section: B Experiments With Real Datamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Hand-Eye Calibration With a Remote Centre of Motion

Pachtrachai

Vasconcelos

Dwyer

et al. 2019

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…While AR has many entertainment uses, it also has possibilities for industry: motor companies such as Toyota, Kia and BMW have developed applications that use AR glasses to assist mechanics in performing automobile maintenance; simulators of robot‐assisted surgery systems based on AR techniques have been developed for training surgeons (Matu et al . ). In addition, many companies, such as iPhone, Google and BlackBerry, have developed apps based on AR that superimpose additional real‐time information on images acquired by smartphones, such as the location of shops, road information and so on (http://www.junaio.com; http://www.augmented-reality.fr/cest-quoi-la-realite-augmentee/des-exemples-dutilisation-de-la-realite-augmentee/; http://www.augmentedrealitytrends.com/).…”

Section: Introductionmentioning

confidence: 97%

ARCube—The Augmented Reality Cube for Archaeology

Fernández-Palacios

Nex

Rizzi³

et al. 2014

Archaeometry

View full text Add to dashboard Cite

Augmented Reality (AR) allows the direct and intuitive access of digital objects visualized in real time on computer screens. AR applications are commonly used in many different areas, such as entertainment, sports and tourism, while their use in archaeology is still limited. When employed, current archaeological AR applications use expensive devices or flat targets, which are insufficient for visualizing complex artefacts. In this paper, we present a low-cost, automated method called the ARCube System, which allows the expansion and enrichment of AR applications focused on archaeological objects. In this paper, the newly developed ARCube System is described by presenting several archaeological examples to show the system's ability to visualize and investigate archaeological finds. The reported tests demonstrate the method's reliability on a variety of objects characterized by different shapes and sizes. The ARCube System allows users to interact with digital 3D models, rotate them through 360°and explore details in high resolution (without any risk of damaging the find). The system renders 1:1 scale between digital and actual object and offers a low-cost flexible system for the interactive visualization of archaeological finds that can be beneficial to public research (e.g., museum exhibits) and education (e.g., the classroom) and may possibly lead to new avenues of archaeological research.

show abstract

“…The doctor could view images using a head-up or head-mounted display (such as HoloLens glasses), which enabled doctors to examine internal structures or lesions through covered tissue. AR also provides the physician with a visually sound anatomical structure [29][30][31][32]. Currently, virtual and real registration technology based on AR is a research hotspot.…”

Section: Introductionmentioning

confidence: 99%

Optimization of virtual and real registration technology based on augmented reality in a surgical navigation system

et al. 2020

View full text Add to dashboard Cite

Background: The traditional navigation interface was intended only for two-dimensional observation by doctors; thus, this interface does not display the total spatial information for the lesion area. Surgical navigation systems have become essential tools that enable for doctors to accurately and safely perform complex operations. The image navigation interface is separated from the operating area, and the doctor needs to switch the field of vision between the screen and the patient's lesion area. In this paper, augmented reality (AR) technology was applied to spinal surgery to provide more intuitive information to surgeons. The accuracy of virtual and real registration was improved via research on AR technology. During the operation, the doctor could observe the AR image and the true shape of the internal spine through the skin. Methods: To improve the accuracy of virtual and real registration, a virtual and real registration technique based on an improved identification method and robotassisted method was proposed. The experimental method was optimized by using the improved identification method. X-ray images were used to verify the effectiveness of the puncture performed by the robot. Results: The final experimental results show that the average accuracy of the virtual and real registration based on the general identification method was 9.73 ± 0.46 mm (range 8.90-10.23 mm). The average accuracy of the virtual and real registration based on the improved identification method was 3.54 ± 0.13 mm (range 3.36-3.73 mm). Compared with the virtual and real registration based on the general identification method, the accuracy was improved by approximately 65%. The highest accuracy of the virtual and real registration based on the robot-assisted method was 2.39 mm. The accuracy was improved by approximately 28.5% based on the improved identification method. Conclusion: The experimental results show that the two optimized methods are highly very effective. The proposed AR navigation system has high accuracy and stability. This system may have value in future spinal surgeries.

show abstract

Stereoscopic augmented reality system for supervised training on minimal invasive surgery robots

Cited by 14 publications

References 3 publications

Hand-Eye Calibration With a Remote Centre of Motion

Hand-Eye Calibration With a Remote Centre of Motion

ARCube—The Augmented Reality Cube for Archaeology

Optimization of virtual and real registration technology based on augmented reality in a surgical navigation system

Contact Info

Product

Resources

About