Applying Depth-Sensing to Automated Surgical Manipulation with a da Vinci Robot

Hwang, Mina; Seita, Daniel; Thananjeyan, Brijen; Ichnowski, Jeffrey; Paradis, Samuel; Fer, Danyal; Low, Thomas; Goldberg, Ken

doi:10.1109/ismr48331.2020.9312948

Cited by 30 publications

(37 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the generation of etalon data, it is necessary to sort the data to evaluate positions and to define the appropriate position of points in the space. This is done mainly by generating the algorithm for sorting points based on (6), summarizing them, and computing the median in the specific position in (8), where "k" represents the number of scans used to generate the etalon data. Variable "n" represents the numbers of A i,j from scans for computing the mean value of etalon data (M i,j ).…”

Section: The Experimental Inspection Stand Based On the Laser Sensormentioning

confidence: 99%

“…The other possibility to gain data from the manufacturing is by implementing commercial solutions based on 3D camera vision, such as Zivid Two or Zivid One+ from Zivid company. The application is for instance described in [8], where it demonstrates the work of the surgical Da Vinci robot with a 3D view on pegboard. Other similar commercial solutions are MotionCam-3D and PhoXi 3D Scanner GEN 2 from the Photoneo company.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Analysis of Laser Sensors and Camera Vision in the Shoe Position Inspection System

Klarák

Kuric

Zajačko

et al. 2021

Sensors

View full text Add to dashboard Cite

Inspection systems are currently an evolving field in the industry. The main goal is to provide a picture of the quality of intermediates and products in the production process. The most widespread sensory system is camera equipment. This article describes the implementation of camera devices for checking the location of the upper on the shoe last. The next part of the article deals with the analysis of the application of laser sensors in this task. The results point to the clear advantages of laser sensors in the inspection task of placing the uppers on the shoe’s last. The proposed method defined the resolution of laser scanners according to the type of scanned surface, where the resolution of point cloud ranged from 0.16 to 0.5 mm per point based on equations representing specific points approximated to polynomial regression in specific places, which are defined in this article. Next, two inspection systems were described, where one included further development in the field of automation and Industry 4.0 and with a high perspective of development into the future. The main aim of this work is to conduct analyses of sensory systems for inspection systems and their possibilities for further work mainly based on the resolution and quality of obtained data. For instance, dependency on scanning complex surfaces and the achieved resolution of scanned surfaces.

show abstract

Section: The Experimental Inspection Stand Based On the Laser Sensormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Analysis of Laser Sensors and Camera Vision in the Shoe Position Inspection System

Klarák

Kuric

Zajačko

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…In [18] an FSM based framework for automation of surgical sub-tasks is developed and tested on simple surgeon training tasks like peg&ring and knot-tying. In [19] a depth-sensor has been used to increase the accuracy of peg&ring task. Finally, in [20], a Hierarchical Finite State Machine (HFSM) is exploited to control autonomous mobile systems.…”

Section: Related Workmentioning

confidence: 99%

Modeling of Surgical Procedures Using Statecharts for Semi-Autonomous Robotic Surgery

Falezza

Piccinelli

Rossi

et al. 2021

IEEE Trans. Med. Robot. Bionics

View full text Add to dashboard Cite

In this paper we propose a new methodology to model surgical procedures that is specifically tailored to semiautonomous robotic surgery. We propose to use a restricted version of statecharts to merge the bottom-up approach, based on data-driven techniques (e.g. machine learning), with the top-down approach based on knowledge representation techniques. We consider medical knowledge about the procedure and sensing of the environment in two concurrent regions of the statecharts to facilitate re-usability and adaptability of the modules. Our approach allows producing a well defined procedural model exploiting the hierarchy capability of the statecharts, while machine learning modules act as soft sensors to trigger state transitions. Integrating data driven and prior knowledge techniques provides a robust, modular, flexible and re-configurable methodology to define a surgical procedure which is comprehensible by both humans and machines. We validate our approach on the three surgical phases of a Robot-Assisted Radical Prostatectomy (RARP) that directly involve the assistant surgeon: bladder mobilization, bladder neck transection, and vesicourethral anastomosis, all performed on synthetic manikins.

show abstract

“…Robotic Surgical Assistants (RSAs) currently rely on human supervision for the entirety of surgical tasks, which can consist of many very repetitive subtasks such as suturing. Automation of surgical subtasks may reduce surgeon fatigue [37], with initial results in surgical cutting [23,34], debridement [23,30], suturing [7-10, 28, 31-33], hemostasis [26], and peg transfer [12,13,24,27].…”

Section: Introductionmentioning

confidence: 99%

“…This handover motion is performed in between stitches during suturing, and is a critical step, as accurately positioning the needle in the end effector affects the stability of its trajectory when guided through tissue. Because varying cable tension in the cables driving the arms causes inaccuracies in motions, a high precision task such as passing a needle between the end effectors is challenging [12,13,21,24,25].…”

Section: Introductionmentioning

confidence: 99%

Learning to Localize, Grasp, and Hand Over Unmodified Surgical Needles

Wilcox¹,

Kerr²,

Thananjeyan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Robotic Surgical Assistants (RSAs) are commonly used to perform minimally invasive surgeries by expert surgeons. However, long procedures filled with tedious and repetitive tasks such as suturing can lead to surgeon fatigue, motivating the automation of suturing. As visual tracking of a thin reflective needle is extremely challenging, prior work has modified the needle with nonreflective contrasting paint. As a step towards automation of a suturing subtask without modifying the needle, we propose HOUSTON: Handoff of Unmodified, Surgical, Tool-Obstructed Needles, a problem and algorithm that uses a learned active sensing policy with a stereo camera to localize and align the needle into a visible and accessible pose for the other arm. To compensate for robot positioning and needle perception errors, the algorithm then executes a high-precision grasping motion that uses multiple cameras. In physical experiments using the da Vinci Research Kit (dVRK), HOUSTON successfully passes unmodified surgical needles with a success rate of 96.7% and is able to perform handover sequentially between the arms 32.4 times on average before failure. On needles unseen in training, HOUSTON achieves a success rate of 75 − 92.9%. To our knowledge, this work is the first to study handover of unmodified surgical needles. See https://tinyurl.com/houston-surgery for additional materials.

show abstract

Applying Depth-Sensing to Automated Surgical Manipulation with a da Vinci Robot

Cited by 30 publications

References 46 publications

Analysis of Laser Sensors and Camera Vision in the Shoe Position Inspection System

Analysis of Laser Sensors and Camera Vision in the Shoe Position Inspection System

Modeling of Surgical Procedures Using Statecharts for Semi-Autonomous Robotic Surgery

Learning to Localize, Grasp, and Hand Over Unmodified Surgical Needles

Contact Info

Product

Resources

About