Combining Augmented Reality and 3D Printing to Improve Surgical Workflows in Orthopedic Oncology: Smartphone Application and Clinical Evaluation

Abstract

During the last decade, orthopedic oncology has experienced the benefits of computerized medical imaging to reduce human dependency, improving accuracy and clinical outcomes. However, traditional surgical navigation systems do not always adapt properly to this kind of interventions. Augmented reality (AR) and three-dimensional (3D) printing are technologies lately introduced in the surgical environment with promising results. Here we present an innovative solution combining 3D printing and AR in orthopedic oncological surgery. A new surgical workflow is proposed, including 3D printed models and a novel AR-based smartphone application (app). This app can display the patient’s anatomy and the tumor’s location. A 3D-printed reference marker, designed to fit in a unique position of the affected bone tissue, enables automatic registration. The system has been evaluated in terms of visualization accuracy and usability during the whole surgical workflow. Experiments on six realistic phantoms provided a visualization error below 3 mm. The AR system was tested in two clinical cases during surgical planning, patient communication, and surgical intervention. These results and the positive feedback obtained from surgeons and patients suggest that the combination of AR and 3D printing can improve efficacy, accuracy, and patients’ experience.

The Use of Mobile Applications for the Diagnosis and Treatment of Tumors in Orthopaedic Oncology – a Systematic Review

“…Three articles focussed on a smartphone application for musculoskeletal tumors in trauma or orthopaedic surgery. Two articles dealt with augmented reality to support tumor resection [7,12]. One was developed to improve the follow-up of patients with musculoskeltal tumors [13].…”

“…Another reason for the still restrained use of smartphones in the daily work of an orthopaedic or trauma surgeon is the poor visibility of relevant mobile applications in App Stores. Several mobile applications described in the included articles were not available under the specified search criteria in App Stores [7,14,22]. When publishing a new application, this should be taken into account.…”

“…The three studies that address smartphone applications in tumor orthopaedics are two technical notes describing the possibility of using augmented reality via the phone's camera to excise a tumor [7,12] and another technical note about a mobile application developed as a post-surgery tracking-system for patients with musculoskeletal tumors [13]. As the three articles are technical notes no analysing of methological quality according to MINOR criteria was performed.…”

“…Nevertheless, the initial diagnosis of musculoskeletal tumors is an everyday challenge for non-specialized physicians in facilities of any level of care. Especially for rare diseases, the support of physicians by mHealth applications is beneficial [7]. The aim of this review is to give orthopaedic or trauma surgeons an overview of currently available literature about mobile medical applications with the focus on the diagnosis and treatment of musculoskeletal tumors and to present available mobile applications.…”

Augmented Reality as a Tool to Guide PSI Placement in Pelvic Tumor Resections

“…Devices such as smartphones or head-mounted displays are commonly used for this purpose. Inside the operating room, AR mainly focuses on visualizing the tumor and other anatomical structures [ 17 , 18 , 19 ], enhancing guidance during resections. The virtual models can be displayed in accordance with the patient’s anatomy by performing a registration step, which is either calculated manually [ 20 ], using tracking systems [ 21 , 22 ], through advanced algorithms for surface registration [ 23 ], or by markers detected with pattern recognition techniques [ 24 ].…”