A Microsoft HoloLens Mixed Reality Surgical Simulator for Patient-Specific Hip Arthroplasty Training

Turini, Giuseppe; Condino, Sara; Parchi, Paolo Domenico; Viglialoro, Rosanna Maria; Piolanti, Nicola; Gesi, Marco; Ferrari, Mauro; Ferrari, Vincenzo

doi:10.1007/978-3-319-95282-6_15

Cited by 22 publications

(13 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, the outcome of the survey has shown that the use of both DICOM Viewer and Spectator View is useful for surgical education purposes. We would highlight that the real use of the conceived platform in the operating room has represented a remarkable feature of our approach, since most if not all studies conducted so far in literature exploit mixed reality only in simulated environments [28] and not in real operating rooms. In conclusion, the authors believe that mixed reality represents a promising technique that in a few years will enter the operating rooms to support surgeons during surgical procedures in many hospitals across the world.…”

Section: Resultsmentioning

confidence: 99%

Experimental Setup Employed in the Operating Room Based on Virtual and Mixed Reality: Analysis of Pros and Cons in Open Abdomen Surgery

Galati

Simone

Barile

et al. 2020

Journal of Healthcare Engineering

View full text Add to dashboard Cite

Currently, surgeons in operating rooms are forced to focus their attention both on the patient’s body and on flat low-quality surgical monitors, in order to get all the information needed to successfully complete surgeries. The way the data are displayed leads to disturbances of the surgeon’s visuals, which may affect his performances, besides the fact that other members of the surgical team do not have proper visual tools able to aid him. The idea underlying this paper is to exploit mixed reality to support surgeons during surgical procedures. In particular, the proposed experimental setup, employed in the operating room, is based on an architecture that put together the Microsoft HoloLens, a Digital Imaging and Communications in Medicine (DICOM) player and a mixed reality visualization tool (i.e., Spectator View) developed by using the Mixed Reality Toolkit in Unity with Windows 10 SDK. The suggested approach enables visual information on the patient’s body as well as information on the results of medical screenings to be visualized on the surgeon’s headsets. Additionally, the architecture enables any data and details to be shared by the team members or by external users during surgical operations. The paper analyses in detail advantages and drawbacks that the surgeons have found when they wore the Microsoft HoloLens headset during all the ten open abdomen surgeries conducted at the IRCCS Hospital “Giovanni Paolo II” in the city of Bari (Italy). A survey based on Likert scale demonstrates how the use of the suggested tools can increase the execution speed by allowing multitasking procedures, i.e., by checking medical images at high resolution without leaving the operating table and the patient. On the other hand, the survey also reveals an increase in the physical stress and reduced comfort due to the weight of the Microsoft HoloLens device, along with drawbacks due to the battery autonomy. Additionally, the survey seems to encourage the use of DICOM Viewer and Spectator View both for surgical education and for improving surgery outcomes. Note that the real use of the conceived platform in the operating room represents a remarkable feature of this paper, since most if not all the studies conducted so far in literature exploit mixed reality only in simulated environments and not in real operating rooms. In conclusion, the study clearly highlights that, despite the challenges required in the forthcoming years to improve the current technology, mixed reality represents a promising technique that will soon enter the operating rooms to support surgeons during surgical procedures in many hospitals across the world.

show abstract

Section: Resultsmentioning

confidence: 99%

Experimental Setup Employed in the Operating Room Based on Virtual and Mixed Reality: Analysis of Pros and Cons in Open Abdomen Surgery

Galati

Simone

Barile

et al. 2020

Journal of Healthcare Engineering

View full text Add to dashboard Cite

show abstract

“…However there is a growing interest also in using commercial OST HMDs to guide manual tasks using VR data which should be accurately aligned to the real target, as in AR guided surgery [11,12]. By way of example: a virtual cutting line/plane registered with the patient anatomy to guide the surgical incision/cutting [13], and a virtual axis displayed over a bone surface for the alignment of a surgical reamer according to a preoperative plan [14]. In such cases, the VR content is superimposed to the physical target, and the visual experience can be impaired by the blurring caused by simultaneously viewing virtual content and physical objects at differing focal distances.…”

mentioning

confidence: 99%

Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks

Condino

Carbone

Piazza

et al. 2020

IEEE Trans. Biomed. Eng.

Self Cite

109

102

View full text Add to dashboard Cite

Objective: The focal length of available optical seethrough (OST) head-mounted displays (HMDs) is at least 2 m therefore, during manual tasks, the user eye cannot keep in focus both the virtual and real content at the same time. Another perceptual limitation is related to the vergence-accommodation conflict (VAC), this latter being present in binocular vision only. This paper investigates the effect of incorrect focus cues on the user performance, visual comfort and workload during the execution of augmented reality (AR) guided manual task with one of the most advanced OST HMD, the Microsoft HoloLens. Methods: An experimental study was designed to investigate the performance of 20 subjects in a connect-the-dots task, with and without the use of AR. The following tests were planned: AR guided monocular and binocular; Naked-eye monocular and binocular. Each trial was analyzed to evaluate the accuracy in connecting dots; NASA Task Load Index and Likert questionnaires were used to assess the workload and the visual comfort. Results: No statistically significant differences were found in the workload, and in the perceived comfort between the AR guided binocular and monocular test. User performances were significantly better during the Naked eye tests. No statistically significant differences in performances were found in the monocular and binocular tests. The maximum error in AR tests was 5.9 mm. Conclusion: Even if there is a growing interest in using commercial OST-HMD, for guiding high-precision manual tasks, attention should be paid to the limitations of the available technology not designed for the peripersonal space.

show abstract

“…The applications were developed using the Mixed Reality Toolkit development kit for Unity, which has become the standard for developing any MR application with HoloLens devices. This has been used in many medical applications for rhinoplasty (Maasthi et al, 2020), arthroplasty training (Turini et al, 2018), and open abdominal surgery (Galati et al, 2020). Loading of preoperative imaging studies with Digital Imaging and Communication on Medicine (DICOM) format was performed using a customised version of the Fellow Oak DICOM toolkit (Al-Zu'bi et al, 2017).…”

Section: Mixed Reality Frameworkmentioning

confidence: 99%

Application of Mixed Reality in Medical Training and Surgical Planning Focused on Minimally Invasive Surgery

Sánchez‐Margallo

Miguel

Anzules

et al. 2021

Front. Virtual Real.

View full text Add to dashboard Cite

Introduction: Medical training is a long and demanding process, in which the first stages are usually based on two-dimensional, static, and unrealistic content. Conversely, advances in preoperative imaging have made it an essential part of any successful surgical procedure. However, access to this information often requires the support of an assistant and may compromise sterility in the surgical process. Herein, we present two solutions based on mixed reality that aim to improve both training and planning in minimally invasive surgery.Materials and Methods: Applications were developed for the use of the Microsoft HoloLens device. The urology training application provided access to a variety of anatomical and surgical training contents. Expert urological surgeons completed a questionnaire to evaluate its use. The surgical planning solution was used during laparoscopic renal tumorectomy in an experimental model and video-assisted right upper lobectomy in an adult patient. Surgeons reported their experience using this preoperative planning tool for surgery.Results: The solution developed for medical training was considered a useful tool for training in urological anatomy, facilitating the translation of this knowledge to clinical practice. Regarding the solution developed for surgical planning, it allowed surgeons to access the patient’s clinical information in real-time, such as preoperative imaging studies, three-dimensional surgical planning models, or medical history, facilitating the surgical approach. The surgeon’s view through the mixed reality device was shared with the rest of the surgical team.Conclusions: The mixed reality-based solution for medical training facilitates the transfer of knowledge into clinical practice. The preoperative planning tool for surgery provides real-time access to essential patient information without losing the sterility of the surgical field. However, further studies are needed to comprehensively validate its clinical application.

show abstract

A Microsoft HoloLens Mixed Reality Surgical Simulator for Patient-Specific Hip Arthroplasty Training

Cited by 22 publications

References 17 publications

Experimental Setup Employed in the Operating Room Based on Virtual and Mixed Reality: Analysis of Pros and Cons in Open Abdomen Surgery

Experimental Setup Employed in the Operating Room Based on Virtual and Mixed Reality: Analysis of Pros and Cons in Open Abdomen Surgery

Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks

Application of Mixed Reality in Medical Training and Surgical Planning Focused on Minimally Invasive Surgery

Contact Info

Product

Resources

About