Needle Tip Force Estimation Using an OCT Fiber and a Fused convGRU-CNN Architecture

Gessert, Nils; Priegnitz, Torben; Saathoff, Thore; Antoni, Sven-Thomas; Meyer, David E.; Hamann, Moritz; Jünemann, Klaus-Peter; Otte, Christoph; Schlaefer, Alexander

doi:10.1007/978-3-030-00937-3_26

Cited by 6 publications

(10 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, we provide qualitative results for tissue insertion experiments, showing the feasibility of the approach. This work extends preliminary results we presented at MICCAI 2018 [27]. We substantially revised and extended the original paper with an extended review of the relevant literature, a more detailed explanation of our novel convGRU-CNN model and additional experiments for the model.…”

Section: Introductionsupporting

confidence: 61%

“…Furthermore, we employ recurrent dropout for additional regularization [29] at the cell input with probability p di = 0.1 and at the cell output with probability p do = 0.2. Recurrent batch normalization and dropout are extensions to the original model presented in [27] and the new model is named convGRU-CNN+. We add these augmentations to all recurrent models.…”

Section: Deep Learning Architecturesmentioning

confidence: 99%

See 1 more Smart Citation

Spatio-temporal deep learning models for tip force estimation during needle insertion

et al. 2019

Self Cite

View full text Add to dashboard Cite

PurposePrecise placement of needles is a challenge in a number of clinical applications such as brachytherapy or biopsy. Forces acting at the needle cause tissue deformation and needle deflection which in turn may lead to misplacement or injury. Hence, a number of approaches to estimate the forces at the needle have been proposed. Yet, integrating sensors into the needle tip is challenging and a careful calibration is required to obtain good force estimates.MethodsWe describe a fiber-optic needle tip force sensor design using a single OCT fiber for measurement. The fiber images the deformation of an epoxy layer placed below the needle tip which results in a stream of 1D depth profiles. We study different deep learning approaches to facilitate calibration between this spatio-temporal image data and the related forces. In particular, we propose a novel convGRU-CNN architecture for simultaneous spatial and temporal data processing.ResultsThe needle can be adapted to different operating ranges by changing the stiffness of the epoxy layer. Likewise, calibration can be adapted by training the deep learning models. Our novel convGRU-CNN architecture results in the lowest mean absolute error of and a cross-correlation coefficient of 0.9997 and clearly outperforms the other methods. Ex vivo experiments in human prostate tissue demonstrate the needle’s application.ConclusionsOur OCT-based fiber-optic sensor presents a viable alternative for needle tip force estimation. The results indicate that the rich spatio-temporal information included in the stream of images showing the deformation throughout the epoxy layer can be effectively used by deep learning models. Particularly, we demonstrate that the convGRU-CNN architecture performs favorably, making it a promising approach for other spatio-temporal learning problems.Electronic supplementary materialThe online version of this article (10.1007/s11548-019-02006-z) contains supplementary material, which is available to authorized users.

show abstract

Section: Introductionsupporting

confidence: 61%

Section: Deep Learning Architecturesmentioning

confidence: 99%

Spatio-temporal deep learning models for tip force estimation during needle insertion

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, biased force measurement due to friction forces at the tool shaft is undesirable. Gessert et al [3] proposed a miniature force sensor integrated in the tool tip. These force sensors can be problematic due to sterilization, biocompatibility and integration in microsurgical instruments with a working channel.…”

Section: Problemmentioning

confidence: 99%

Force estimation from 4D OCT data in a human tumor xenograft mouse model

Neidhardt

Gessert

Gosau

et al. 2020

Current Directions in Biomedical Engineering

Self Cite

View full text Add to dashboard Cite

Minimally invasive robotic surgery offer benefits such as reduced physical trauma, faster recovery and lesser pain for the patient. For these procedures, visual and haptic feedback to the surgeon is crucial when operating surgical tools without line-of-sight with a robot. External force sensors are biased by friction at the tool shaft and thereby cannot estimate forces between tool tip and tissue. As an alternative, vision-based force estimation was proposed. Here, interaction forces are directly learned from deformation observed by an external imaging system. Recently, an approach based on optical coherence tomography and deep learning has shown promising results. However, most experiments are performed on ex-vivo tissue. In this work, we demonstrate that models trained on dead tissue do not perform well in in vivo data. We performed multiple experiments on a human tumor xenograft mouse model, both on in vivo, perfused tissue and dead tissue. We compared two deep learning models in different training scenarios. Training on perfused, in vivo data improved model performance by 24% for in vivo force estimation.

show abstract

“…The diameter of the needle is 2 mm. In [2], the needle tip was constructed as a cone and attached to the needle shaft with a deformable epoxy layer. Thus, radial forces on the needle tip could easily tilt it.…”

Section: Needle Designmentioning

confidence: 99%

“…Here, we consider a setting where optical coherence tomography is B M. Gromniak martin.gromniak@tuhh.de 1 Institute of Medical Technology, Hamburg University of Technology, Hamburg, Germany available, e.g., to study tissue deformation [10] or to realize elastography [7]. While OCT has been proposed for tip forces estimation before [2,3], these approaches rely on the reconstructed gray value data. However, using the reconstructed data has two limitations.…”

Section: Introductionmentioning

confidence: 99%

Needle tip force estimation by deep learning from raw spectral OCT data

et al. 2020

View full text Add to dashboard Cite

Purpose Needle placement is a challenging problem for applications such as biopsy or brachytherapy. Tip force sensing can provide valuable feedback for needle navigation inside the tissue. For this purpose, fiber-optical sensors can be directly integrated into the needle tip. Optical coherence tomography (OCT) can be used to image tissue. Here, we study how to calibrate OCT to sense forces, e.g., during robotic needle placement. Methods We investigate whether using raw spectral OCT data without a typical image reconstruction can improve a deep learning-based calibration between optical signal and forces. For this purpose, we consider three different needles with a new, more robust design which are calibrated using convolutional neural networks (CNNs). We compare training the CNNs with the raw OCT signal and the reconstructed depth profiles. Results We find that using raw data as an input for the largest CNN model outperforms the use of reconstructed data with a mean absolute error of 5.81 mN compared to 8.04 mN. Conclusions We find that deep learning with raw spectral OCT data can improve learning for the task of force estimation. Our needle design and calibration approach constitute a very accurate fiber-optical sensor for measuring forces at the needle tip.

show abstract

Needle Tip Force Estimation Using an OCT Fiber and a Fused convGRU-CNN Architecture

Cited by 6 publications

References 16 publications

Spatio-temporal deep learning models for tip force estimation during needle insertion

Spatio-temporal deep learning models for tip force estimation during needle insertion

Force estimation from 4D OCT data in a human tumor xenograft mouse model

Needle tip force estimation by deep learning from raw spectral OCT data

Contact Info

Product

Resources

About