i3PosNet: instrument pose estimation from X-ray in temporal bone surgery

Kügler, David; Sehring, Jannik; Stefanov, Andrei; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Mukhopadhyay, Anirban

doi:10.1007/s11548-020-02157-4

Cited by 28 publications

(30 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To guide surgeons and robotic instruments in image-guided temporal bone surgery, instrument poses need to be estimated with high-precision. Since the direct prediction of poses from full images is difficult, the state-of-the-art modular framework i3PosNet [7] implements "CROP" and "POSE" operations (see Fig. 1a).…”

Section: Problem Definition Of Pose Estimationmentioning

confidence: 99%

“…We use the publicly available i3PosNet Dataset [7] assuming its naming conventions. Dataset A features synthetic and Dataset C real radiographs.…”

Section: Training and Evaluation Datasetsmentioning

confidence: 99%

“…This CAI-centric challenge is featured in instrument pose estimation for minimally-invasive temporal bone surgery [12]: Training on synthetic data is necessary, because hard-to-acquire real-world datasets with high-quality annotation are reserved for evaluation only. The state-of-the-art method (i3PosNet [7]) relies on an architecture optimized for classification. It disregards the specialization potential as described by the no-free-lunch-theorem and as realized by DNNs for registration [2] demanding a method to automatically improve the architecture.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

AutoSNAP: Automatically Learning Neural Architectures for Instrument Pose Estimation

Kügler¹,

Uecker²,

Kuijper³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Despite recent successes, the advances in Deep Learning have not yet been fully translated to Computer Assisted Intervention (CAI) problems such as pose estimation of surgical instruments. Currently, neural architectures for classification and segmentation tasks are adopted ignoring significant discrepancies between CAI and these tasks. We propose an automatic framework (AutoSNAP) for instrument pose estimation problems, which discovers and learns architectures for neural networks. We introduce 1) an efficient testing environment for pose estimation, 2) a powerful architecture representation based on novel Symbolic Neural Architecture Patterns (SNAPs), and 3) an optimization of the architecture using an efficient search scheme. Using AutoSNAP, we discover an improved architecture (SNAPNet) which outperforms both the hand-engineered i3PosNet and the state-of-the-art architecture search method DARTS.

show abstract

Section: Problem Definition Of Pose Estimationmentioning

confidence: 99%

“…We use the publicly available i3PosNet Dataset [7] assuming its naming conventions. Dataset A features synthetic and Dataset C real radiographs.…”

Section: Training and Evaluation Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

AutoSNAP: Automatically Learning Neural Architectures for Instrument Pose Estimation

Kügler¹,

Uecker²,

Kuijper³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…obtained on synthetic data generalize to clinical cases. Most of the previous works [1,14,15,32] use straightforward ray casting for image formation, which is very fast and allows for the generation of arbitrarily large datasets. Ray casting mimics mono-energetic Xray sources passing through objects that consist of a single material, an assumption that undoubtedly is violated in clinical X-ray imaging.…”

Section: Related Workmentioning

confidence: 99%

“…As a consequence, the synthetic images do not exhibit the characteristic artifacts encountered in practice, such as beam hardening and complex noise characteristics. This shortcoming raises questions regarding performance on real data, with several manuscripts limiting evaluation to synthetic images generated using the same tools [14,32,37] or reporting poor generalizability when applied to clinical data [34,42].…”

Section: Related Workmentioning

confidence: 99%

Enabling machine learning in X-ray-based procedures via realistic simulation of image formation

et al. 2019

View full text Add to dashboard Cite

Purpose-Machine learning-based approaches now outperform competing methods in most disciplines relevant to diagnostic radiology. Image-guided procedures, however, have not yet benefited substantially from the advent of deep learning, in particular because images for procedural guidance are not archived and thus unavailable for learning, and even if they were available, annotations would be a severe challenge due to the vast amounts of data. In silico simulation of X-ray images from 3D CT is an interesting alternative to using true clinical radiographs since labeling is comparably easy and potentially readily available.Methods-We extend our framework for fast and realistic simulation of fluoroscopy from highresolution CT, called DeepDRR, with tool modeling capabilities. The framework is publicly available, open source, and tightly integrated with the software platforms native to deep learning, i.e., Python, PyTorch, and PyCuda. DeepDRR relies on machine learning for material decomposition and scatter estimation in 3D and 2D, respectively, but uses analytic forward projection and noise injection to ensure acceptable computation times. On two X-ray image analysis tasks, namely (1) anatomical landmark detection and (2) segmentation and localization of robot end-effectors, we demonstrate that convolutional neural networks (ConvNets) trained on DeepDRRs generalize well to real data without re-training or domain adaptation. To this end, we use the exact same training protocol to train ConvNets on naïve and DeepDRRs and compare their performance on data of cadaveric specimens acquired using a clinical C-arm X-ray system.

show abstract

Artificial Intelligence in Temporal Bone Imaging: A Systematic Review

Spinos,

Martinos,

Petsiou

et al. 2024

The Laryngoscope

View full text Add to dashboard Cite

ObjectiveThe human temporal bone comprises more than 30 identifiable anatomical components. With the demand for precise image interpretation in this complex region, the utilization of artificial intelligence (AI) applications is steadily increasing. This systematic review aims to highlight the current role of AI in temporal bone imaging.Data SourcesA Systematic Review of English Publications searching MEDLINE (PubMed), COCHRANE Library, and EMBASE.Review MethodsThe search algorithm employed consisted of key items such as ‘artificial intelligence,’ ‘machine learning,’ ‘deep learning,’ ‘neural network,’ ‘temporal bone,’ and ‘vestibular schwannoma.’ Additionally, manual retrieval was conducted to capture any studies potentially missed in our initial search. All abstracts and full texts were screened based on our inclusion and exclusion criteria.ResultsA total of 72 studies were included. 95.8% were retrospective and 88.9% were based on internal databases. Approximately two‐thirds involved an AI‐to‐human comparison. Computed tomography (CT) was the imaging modality in 54.2% of the studies, with vestibular schwannoma (VS) being the most frequent study item (37.5%). Fifty‐eight out of 72 articles employed neural networks, with 72.2% using various types of convolutional neural network models. Quality assessment of the included publications yielded a mean score of 13.6 ± 2.5 on a 20‐point scale based on the CONSORT‐AI extension.ConclusionCurrent research data highlight AI's potential in enhancing diagnostic accuracy with faster results and decreased performance errors compared to those of clinicians, thus improving patient care. However, the shortcomings of the existing research, often marked by heterogeneity and variable quality, underscore the need for more standardized methodological approaches to ensure the consistency and reliability of future data.Level of EvidenceNA Laryngoscope, 2024

show abstract

i3PosNet: instrument pose estimation from X-ray in temporal bone surgery

Cited by 28 publications

References 30 publications

AutoSNAP: Automatically Learning Neural Architectures for Instrument Pose Estimation

AutoSNAP: Automatically Learning Neural Architectures for Instrument Pose Estimation

Enabling machine learning in X-ray-based procedures via realistic simulation of image formation

Artificial Intelligence in Temporal Bone Imaging: A Systematic Review

Contact Info

Product

Resources

About