A surgical system for automatic registration, stiffness mapping and dynamic image overlay

Abstract

In this paper we develop a surgical system using the da Vinci research kit (dVRK) that is capable of autonomously searching for tumors and dynamically displaying the tumor location using augmented reality. Such a system has the potential to quickly reveal the location and shape of tumors and visually overlay that information to reduce the cognitive overload of the surgeon. We believe that our approach is one of the first to incorporate state-of-the-art methods in registration, force sensing and tumor localization into a unified surgical system. First, the preoperative model is registered to the intra-operative scene using a Bingham distribution-based filtering approach. An active level set estimation is then used to find the location and the shape of the tumors. We use a recently developed miniature force sensor to perform the palpation. The estimated stiffness map is then dynamically overlaid onto the registered preoperative model of the organ. We demonstrate the efficacy of our system by performing experiments on phantom prostate models with embedded stiff inclusions.

Simultaneous Depth Estimation and Surgical Tool Segmentation in Laparoscopic Images

“…Instrument segmentation may also use supervised or unsupervised methods [6]. Knowing the tissue depth and the instrument masks could facilitate tissue scanning [7] or dynamic image overlays [8], which are useful for laparoscopic surgery.…”

Toward Robotically Automated Femoral Vascular Access

“…These systems have also been extended to automate certain repetitive tasks to reduce the cognitive burden on surgeons [13,25,30,31]. In many of these cases teleoperated systems are adapted to increase the level of automation, moving towards more supervised robotic surgeries.…”

A variable-impedance piezoelectric tactile sensor with tunable sensing performance for tissue hardness sensing in robotic tumor palpation

“…Mechanical properties of anatomy such as tissue hardness extracted from the acquired tactile information are critical for surgeon to make diagnosis of tumors, which are usually significantly stiffer than normal tissues [5]. In addition, palpation also allows surgeon to estimate geometrical information of the tumor site including its size, shape, orientation, etc, which can be used to improve the accuracy and precision in tumor excision tasks [6]. In order to restore the lost sense of touch to RMIS systems, tactile sensors of various principles have been developed, such as capacitive [7], resistive [8][9][10], piezoelectric [11,12], piezoresistive [13], optical [14], fluidic [15], magnetic [16], and also hybrid [17,18].…”