Vision-based deformation recovery for intraoperative force estimation of tool–tissue interaction for neurosurgery

Providing force feedback as relevant information in current Robot-Assisted Minimally Invasive Surgery systems constitutes a technological challenge due to the constraints imposed by the surgical environment. In this context, Sensorless Force Estimation techniques represent a potential solution, enabling to sense the interaction forces between the surgical instruments and soft-tissues. Specifically, if visual feedback is available for observing soft-tissues' deformation, this feedback can be used to estimate the forces applied to these tissues. To this end, a force estimation model, based on Convolutional Neural Networks and Long-Short Term Memory networks, is proposed in this work. This model is designed to process both, the spatiotemporal information present in video sequences and the temporal structure of tool data (the surgical tool-tip trajectory and its grasping status). A series of analyses are carried out to reveal the advantages of the proposal and the challenges that remain for real applications. This research work focuses on two surgical task scenarios, referred to as pushing and pulling tissue. For these two scenarios, different input data modalities and their effect on the force estimation quality are investigated. These input data modalities are tool data, video sequences and a combination of both. The results suggest that the force estimation quality is better when both, the tool data and video sequences, are processed by the neural network model. Moreover, this study reveals the need for a loss function, designed to promote the modeling of smooth and sharp details found in force signals. Finally, the results show that the modeling of forces due to pulling tasks is more challenging than for the simplest pushing actions.

show abstract

“…[25] and [28]), or stereo-correspondences (i.e. [26]). Second, the estimation of forces has been studied only for pushing tasks.…”

Section: Vision-based Force Sensingmentioning

confidence: 99%

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

Marbán

Srinivasan

Samek

et al. 2019

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

show abstract

“…Depending on the tissue type, the required contact force can be under 1.0N [11]. The reduction of the friction is essential to increase the forces sensitivity of the system, especially for the small contact forces required for proper EM imaging.…”

Section: Force Calculation and Sensitivitymentioning

confidence: 99%

A cable-driven parallel manipulator with force sensing capabilities for high-accuracy tissue endomicroscopy

2018

View full text Add to dashboard Cite

PurposeEndomicroscopy (EM) provides high resolution, non-invasive histological tissue information and can be used for scanning of large areas of tissue to assess cancerous and pre-cancerous lesions and their margins. However, current robotic solutions do not provide the accuracy and force sensitivity required to perform safe and accurate tissue scanning.MethodsA new surgical instrument has been developed that uses a cable-driven parallel mechanism (CPDM) to manipulate an EM probe. End-effector forces are determined by measuring the tensions in each cable. As a result, the instrument allows to accurately apply a contact force on a tissue, while at the same time offering high resolution and highly repeatable probe movement.Results0.2 and 0.6 N force sensitivities were found for 1 and 2 DoF image acquisition methods, respectively. A back-stepping technique can be used when a higher force sensitivity is required for the acquisition of high quality tissue images. This method was successful in acquiring images on ex vivo liver tissue.ConclusionThe proposed approach offers high force sensitivity and precise control, which is essential for robotic EM. The technical benefits of the current system can also be used for other surgical robotic applications, including safe autonomous control, haptic feedback and palpation.Electronic supplementary materialThe online version of this article (10.1007/s11548-018-1717-7) contains supplementary material, which is available to authorized users.

show abstract

“…Similarly, polynomial root solving is required for calculating the shortest distance between circles in R 3 or the overlap area of ellipses in R 2 , [19]. It is a well investigated problem, whose solution is crucial to many robotic applications from proximity queries, to imaging [20], and path planning [19].…”

Section: Roots Of Polynomialsmentioning

confidence: 99%

Efficient Proximity Queries for Continuum Robots on Parallel Computing Hardware

Leibrandt

Yang

2017

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Abstract-Surgical manipulators are increasingly capable of approaching deep seated pathologies through convoluted pathways due to advances in the field of continuum robotics. This class of robots can be, in most cases, accurately modelled as a chain of cylindrical shapes. In order to safely and seamlessly telemanipulate these robots, which have complex non-intuitive kinematics, haptic guidance schemes have been developed that rely on accurate proximity queries (PQ) to calculate the distance between the continuum robot and the anatomy. This paper introduces an approach to accurately model continuum robots using cylindrical shaped segments with spherical or flat caps and then efficiently calculate the shortest distance to a triangle mesh. Implementations of efficient, analytical narrow phase PQ calculations for simple and complex geometrical primitives suitable for parallel computing hardware are presented together with experimental validation which show improved performance suitable for real-time robotic applications. An in silico experiment comparing various root-finding algorithms, which are commonly used in PQ calculation or other optimization tasks are compared to assess their suitability when executed on different hardware. The implications of these experimental results are discussed, in particular with regards to the selection of a suitable proximity query algorithm depending on the available parallel computing hardware. Finally, an outlook of future improvements for dense dynamic anatomies is presented.

show abstract

Vision-based deformation recovery for intraoperative force estimation of tool–tissue interaction for neurosurgery

Cited by 39 publications

References 16 publications

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

A cable-driven parallel manipulator with force sensing capabilities for high-accuracy tissue endomicroscopy

Efficient Proximity Queries for Continuum Robots on Parallel Computing Hardware

Contact Info

Product

Resources

About