Estimation of Tool-Tissue Forces in Robot-Assisted Minimally Invasive Surgery Using Neural Networks

Abeywardena, Sajeeva; Yuan, Qiaodi; Tzemanaki, Antonia; Psomopoulou, Efi; Δρούκας, Λεωνίδας; Chris, Melhuish; Dogramadzi, Sanja

doi:10.3389/frobt.2019.00056

Cited by 22 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are, however, limitations to this method due to factors such as lighting conditions. Another solution is force estimation, which utilizes information such as known robot dynamics, motor current, and encoder data to eliminate dealing with force sensors [ 35 ]. However, this approach needs to be improved in terms of accuracy when compared to sensor-based approaches.…”

Section: Overview Of the Robotic Teleoperation Surgery Approachesmentioning

confidence: 99%

A Force-Feedback Methodology for Teleoperated Suturing Task in Robotic-Assisted Minimally Invasive Surgery

Ehrampoosh

Shirinzadeh

Pinskier

et al. 2022

Sensors

View full text Add to dashboard Cite

With robotic-assisted minimally invasive surgery (RAMIS), patients and surgeons benefit from a reduced incision size and dexterous instruments. However, current robotic surgery platforms lack haptic feedback, which is an essential element of safe operation. Moreover, teleportation control challenges make complex surgical tasks like suturing more time-consuming than those that use manual tools. This paper presents a new force-sensing instrument that semi-automates the suturing task and facilitates teleoperated robotic manipulation. In order to generate the ideal needle insertion trajectory and pass the needle through its curvature, the end-effector mechanism has a rotating degree of freedom. Impedance control was used to provide sensory information about needle–tissue interaction forces to the operator using an indirect force estimation approach based on data-based models. The operator’s motion commands were then regulated using a hyperplanar virtual fixture (VF) designed to maintain the desired distance between the end-effector and tissue surface while avoiding unwanted contact. To construct the geometry of the VF, an optoelectronic sensor-based approach was developed. Based on the experimental investigation of the hyperplane VF methodology, improved needle–tissue interaction force, manipulation accuracy, and task completion times were demonstrated. Finally, experimental validation of the trained force estimation models and the perceived interaction forces by the user was conducted using online data, demonstrating the potential of the developed approach in improving task performance.

show abstract

Section: Overview Of the Robotic Teleoperation Surgery Approachesmentioning

confidence: 99%

A Force-Feedback Methodology for Teleoperated Suturing Task in Robotic-Assisted Minimally Invasive Surgery

Ehrampoosh

Shirinzadeh

Pinskier

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…In RAMIS research, there are typically integrated/employed sensors which are not directly related to NTS and workload assessment, but in most of the cases, their modalities show correlation with technical skills [ 17 ]. These sensor types include, but are not limited to, the following devices [ 120 , 121 , 122 , 123 ]: force sensors (strain gauges, capacitive sensors, piezoelectric sensors, optical sensors); tool position sensing (optical, electromagnetic); master/surgeon arm position sensing (external); wearable eyeglasses (Oculus Rift, Google Glass); tool thermal sensor; pressure sensors; camera (RGBD, external); communication (RF sensors); speech (microspeaker); sound (microphones). …”

Section: Technical Approaches For Non-technical Skill and Mental Workload Assessment In Ramismentioning

confidence: 99%

“…As a summary, the following sensors/imaging techniques were studied in NTS and workload assessment in RAMIS (detailed in In RAMIS research, there are typically integrated/employed sensors which are not directly related to NTS and workload assessment, but in most of the cases, their modalities show correlation with technical skills [17]. These sensor types include, but are not limited to, the following devices [120][121][122][123] Automated, sensory data-based NTS and workload assessment can be a key to an objective, reproducible approach to measure the surgeon's skills without bias and the need of human resources. However, these techniques are typically costly, harder to implement and the usage of additional digital tools can be a problem in a clinical environment, even in an Internet of Things setup.…”

Section: Non-technical Skill Assessment-expert Ratingmentioning

confidence: 99%

Non-Technical Skill Assessment and Mental Load Evaluation in Robot-Assisted Minimally Invasive Surgery

Elek

Haidegger

2021

Sensors

View full text Add to dashboard Cite

BACKGROUND: Sensor technologies and data collection practices are changing and improving quality metrics across various domains. Surgical skill assessment in Robot-Assisted Minimally Invasive Surgery (RAMIS) is essential for training and quality assurance. The mental workload on the surgeon (such as time criticality, task complexity, distractions) and non-technical surgical skills (including situational awareness, decision making, stress resilience, communication, leadership) may directly influence the clinical outcome of the surgery. METHODS: A literature search in PubMed, Scopus and PsycNet databases was conducted for relevant scientific publications. The standard PRISMA method was followed to filter the search results, including non-technical skill assessment and mental/cognitive load and workload estimation in RAMIS. Publications related to traditional manual Minimally Invasive Surgery were excluded, and also the usability studies on the surgical tools were not assessed. RESULTS: 50 relevant publications were identified for non-technical skill assessment and mental load and workload estimation in the domain of RAMIS. The identified assessment techniques ranged from self-rating questionnaires and expert ratings to autonomous techniques, citing their most important benefits and disadvantages. CONCLUSIONS: Despite the systematic research, only a limited number of articles was found, indicating that non-technical skill and mental load assessment in RAMIS is not a well-studied area. Workload assessment and soft skill measurement do not constitute part of the regular clinical training and practice yet. Meanwhile, the importance of the research domain is clear based on the publicly available surgical error statistics. Questionnaires and expert-rating techniques are widely employed in traditional surgical skill assessment; nevertheless, recent technological development in sensors and Internet of Things-type devices show that skill assessment approaches in RAMIS can be much more profound employing automated solutions. Measurements and especially big data type analysis may introduce more objectivity and transparency to this critical domain as well. SIGNIFICANCE: Non-technical skill assessment and mental load evaluation in Robot-Assisted Minimally Invasive Surgery is not a well-studied area yet; while the importance of this domain from the clinical outcome’s point of view is clearly indicated by the available surgical error statistics.

show abstract

“…In Sharifi et al (2015) a robot position controller is implemented by exploiting the interaction force estimation between the robotic tool and a soft tissue. In Abeywardena et al (2019) a Neural Networks approach is proposed to map the interaction forces between the robot and a soft environment, exploiting motor current measurements. In Mendizabal et al (2019) a Neural Network approach to classify force ranges from optical coherence tomography (OCT) images is proposed.…”

Section: Related Workmentioning

confidence: 99%

Robust state dependent Riccati equation variable impedance control for robotic force-tracking tasks

Roveda

Piga

2020

Int J Intell Robot Appl

View full text Add to dashboard Cite

Industrial robots are increasingly used in highly flexible interaction tasks, where the intrinsic variability makes difficult to pre-program the manipulator for all the different scenarios. In such applications, interaction environments are commonly (partially) unknown to the robot, requiring the implemented controllers to take in charge for the stability of the interaction. While standard controllers are sensor-based, there is a growing need to make sensorless robots (i.e., most of the commercial robots are not equipped with force/torque sensors) able to sense the environment, properly reacting to the established interaction. This paper proposes a new methodology to sensorless force control manipulators. On the basis of sensorless Cartesian impedance control, an Extended Kalman Filter (EKF) is designed to estimate the interaction exchanged between the robot and the environment. Such an estimation is then used in order to close a robust high-performance force loop, designed exploiting a variable impedance control and a State Dependent Riccati Equation (SDRE) force controller. The described approach has been validated in simulations. A Franka EMIKA panda robot has been considered as a test platform. A probing task involving different materials (i.e., with different stiffness properties) has been considered to show the capabilities of the developed EKF (able to converge with limited errors) and controller (preserving stability and avoiding overshoots). The proposed controller has been compared with an LQR controller to show its improved performance.

show abstract

Estimation of Tool-Tissue Forces in Robot-Assisted Minimally Invasive Surgery Using Neural Networks

Cited by 22 publications

References 13 publications

A Force-Feedback Methodology for Teleoperated Suturing Task in Robotic-Assisted Minimally Invasive Surgery

A Force-Feedback Methodology for Teleoperated Suturing Task in Robotic-Assisted Minimally Invasive Surgery

Non-Technical Skill Assessment and Mental Load Evaluation in Robot-Assisted Minimally Invasive Surgery

Robust state dependent Riccati equation variable impedance control for robotic force-tracking tasks

Contact Info

Product

Resources

About