Modeling elastic objects with neural networks for vision-based force measurement

Greminger, Michael A.; Nelson, Bradley J.

doi:10.1109/iros.2003.1248821

Cited by 23 publications

(20 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…biological tissues) is available from visual feedback (i.e. video sequences), this feedback can be used to estimate the forces applied on these objects, [20] [21]. VBFS methods are developed to estimate forces in 2D or 3D scenarios.…”

Section: Vision-based Force Sensingmentioning

confidence: 99%

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

Marbán

Srinivasan

Samek

et al. 2019

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

Providing force feedback as relevant information in current Robot-Assisted Minimally Invasive Surgery systems constitutes a technological challenge due to the constraints imposed by the surgical environment. In this context, Sensorless Force Estimation techniques represent a potential solution, enabling to sense the interaction forces between the surgical instruments and soft-tissues. Specifically, if visual feedback is available for observing soft-tissues' deformation, this feedback can be used to estimate the forces applied to these tissues. To this end, a force estimation model, based on Convolutional Neural Networks and Long-Short Term Memory networks, is proposed in this work. This model is designed to process both, the spatiotemporal information present in video sequences and the temporal structure of tool data (the surgical tool-tip trajectory and its grasping status). A series of analyses are carried out to reveal the advantages of the proposal and the challenges that remain for real applications. This research work focuses on two surgical task scenarios, referred to as pushing and pulling tissue. For these two scenarios, different input data modalities and their effect on the force estimation quality are investigated. These input data modalities are tool data, video sequences and a combination of both. The results suggest that the force estimation quality is better when both, the tool data and video sequences, are processed by the neural network model. Moreover, this study reveals the need for a loss function, designed to promote the modeling of smooth and sharp details found in force signals. Finally, the results show that the modeling of forces due to pulling tasks is more challenging than for the simplest pushing actions.

show abstract

Section: Vision-based Force Sensingmentioning

confidence: 99%

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

Marbán

Srinivasan

Samek

et al. 2019

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

show abstract

“…Similar methods relying on mechanical deformation models have been studied for MIS scenarios [13,14,15]. Also, learning forces from image information using neural networks has been proposed [16]. More recent approaches have combined template matching and machine learning models [17,18].…”

Section: Introductionmentioning

confidence: 99%

Force estimation from OCT volumes using 3D CNNs

2018

View full text Add to dashboard Cite

show abstract

“…Thereby, scientific community has reported works (e.g. [8][9][10][11] in which combination of visual information and machine learning have offered a good tradeoff between accuracy and computational cost.…”

Section: Introductionmentioning

confidence: 99%

“…[8][9][10] However, most of them have used conventional neural networks which are not enough for RAMIS scenarios due to the vanishing gradient problem, where error signals exhibit exponential decay as they back-propagate through time. In order to avoid this problem, most recently Aviles et al 11 have proposed a solution which combines the richness of the visual information and geometry of motion together with a deep network.…”

Section: Introductionmentioning

confidence: 99%

Exploring the effects of dimensionality reduction in deep networks for force estimation in robotic-assisted surgery

Avilés

Alsaleh

Sobrevilla

et al. 2016

Medical Imaging 2016: Image-Guided Procedures, Robotic Interventions, and Modeling

View full text Add to dashboard Cite

Robotic-Assisted Surgery approach overcomes the limitations of the traditional laparoscopic and open surgeries. However, one of its major limitations is the lack of force feedback. Since there is no direct interaction between the surgeon and the tissue, there is no way of knowing how much force the surgeon is applying which can result in irreversible injuries. The use of force sensors is not practical since they impose different constraints. Thus, we make use of a neuro-visual approach to estimate the applied forces, in which the 3D shape recovery together with the geometry of motion are used as input to a deep network based on LSTM-RNN architecture. When deep networks are used in real time, pre-processing of data is a key factor to reduce complexity and improve the network performance. A common pre-processing step is dimensionality reduction which attempts to eliminate redundant and insignificant information by selecting a subset of relevant features to use in model construction. In this work, we show the effects of dimensionality reduction in a real-time application: estimating the applied force in Robotic-Assisted Surgeries. According to the results, we demonstrated positive effects of doing dimensionality reduction on deep networks including: faster training, improved network performance, and overfitting prevention. We also show a significant accuracy improvement, ranging from about 33% to 86%, over existing approaches related to force estimation.

show abstract

Modeling elastic objects with neural networks for vision-based force measurement

Cited by 23 publications

References 10 publications

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

A recurrent convolutional neural network approach for sensorless force estimation in robotic surgery

Force estimation from OCT volumes using 3D CNNs

Exploring the effects of dimensionality reduction in deep networks for force estimation in robotic-assisted surgery

Contact Info

Product

Resources

About