Computer-Assisted Surgery: Virtual- and Augmented-Reality Displays for Navigation During Planning and Performing Surgery on Large Joints

Abstract

Today, a small number of full-fledged multifunctional CT systems are used for planning intraoperative intervention using mixed reality, for example, in cardiovascular surgery and urology, especially in Oncology. Surgeons today need a wide field of view to operate. Currently, the use of technologies like the DaVinci robot surgeon is constrained by the bulkiness of software and hardware solutions, the lack of trained specialists, and the high cost of equipment. There is no technology for navigating the course of the operation, with a layer of mixed reality being applied to the patient during the operation, so that the surgeon can use it to track the position of the organ, organ systems, and surgical instruments in real-time. The developed software prototype allows you to create three-dimensional models of internal organs based on computer and magnetic resonance imaging images with the possibility of simulating surgical intervention. Augmented reality glasses are also used to practice the skills of operations on a virtual patient using phantom dummies, linking surgical instruments to holography, as well as fully combining the simulation of the preoperative period with a real patient in real-time.