Localization and Control of Magnetic Suture Needles in Cluttered Surgical Site with Blood and Tissue

Abstract

Real-time visual localization of needles is necessary for various surgical applications, including surgical automation and visual feedback. In this study we investigate localization and autonomous robotic control of needles in the context of our magneto-suturing system. Our system holds the potential for surgical manipulation with the benefit of minimal invasiveness and reduced patient side effects. However, the nonlinear magnetic fields produce unintuitive forces and demand delicate position-based control that exceeds the capabilities of direct human manipulation. This makes automatic needle localization a necessity. Our localization method combines neural network-based segmentation and classical techniques, and we are able to consistently locate our needle with 0.73 mm RMS error in clean environments and 2.72 mm RMS error in challenging environments with blood and occlusion. The average localization RMS error is 2.16 mm for all environments we used in the experiments. We combine this localization method with our closed-loop feedback control system to demonstrate the further applicability of localization to autonomous control. Our needle is able to follow a running suture path in (1) no blood, no tissue; (2) heavy blood, no tissue; (3) no blood, with tissue; and (4) heavy blood, with tissue environments. The tip position tracking error ranges from 2.6 mm to 3.7 mm RMS, opening the door towards autonomous suturing tasks.

Overcoming the Force Limitations of Magnetic Robotic Surgery: Magnetic Pulse Actuated Collisions for Tissue‐Penetrating‐Needle for Tetherless Interventions

“…We envision that in vivo experiments would add additional challenges with respect to coordinate transformation, 3D localization, and localization challenges in dynamic and cluttered surgical environments. [25] Focusing on the challenges related to 1) the safe and precise control of the impact mechanism in 3D environments; 2) intraoperative imaging for feedback to the surgeon and for autonomous control applications; and 3) in vivo demonstrations of the MPACT-Needle is the crucial next step to translate our studies into preclinical steps.…”

“…Therefore, a complete penetration from outside of the sample with large penetration distances remains as a challenge for millisizedminiature magnetic needles, as shown in our previous study. [25] Recent studies have demonstrated the advantage of using magnetic impulse impacts to generate large instantaneous forces that suffice for various tissue penetration applications. [26][27][28] The fundamental enabling principle in these studies is to magnetically slide a magnetic element inside of a larger container, allowing the magnetic element to gain sufficient velocity such that, upon striking an impact plate on one side of the container, the impact transfers the sliding magnet's momentum to the impact plate, inducing a momentary force on the entire device or at the needle tip.…”