DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures

Unberath, Mathias; Zaech, Jan Nico; Lee, Sing Chun; Bier, Bastian; Fotouhi, Javad; Armand, Mehran; Navab, Nassir

doi:10.1007/978-3-030-00937-3_12

Cited by 90 publications

(79 citation statements)

References 15 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One example to do so is the deep scatter estimation by Maier et al [96]. Unberath et al drive this even further to emulate the complete X-ray formation process in their DeepDRR [97]. In [98] Horger et al demonstrate that even noise of unknown distributions can be learned, leading to an efficient generative noise model for realistic physical simulations.…”

Section: Physical Simulationmentioning

confidence: 99%

A gentle introduction to deep learning in medical image processing

Maier

Syben

Lasser

et al. 2019

Zeitschrift für Medizinische Physik

417

224

View full text Add to dashboard Cite

This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and neural networks, along with some fundamental theory that is often omitted. Doing so allows us to understand the reasons for the rise of deep learning in many application domains. Obviously medical image processing is one of these areas which has been largely affected by this rapid progress, in particular in image detection and recognition, image segmentation, image registration, and computer-aided diagnosis. There are also recent trends in physical simulation, modelling, and reconstruction that have led to astonishing results. Yet, some of these approaches neglect prior knowledge and hence bear the risk of producing implausible results. These apparent weaknesses highlight current limitations of deep learning. However, we also briefly discuss promising approaches that might be able to resolve these problems in the future.

show abstract

Section: Physical Simulationmentioning

confidence: 99%

A gentle introduction to deep learning in medical image processing

Maier

Syben

Lasser

et al. 2019

Zeitschrift für Medizinische Physik

417

224

View full text Add to dashboard Cite

show abstract

“…Training: Training is performed on realistic digitally reconstructed radiographs (DRRs) generated from CT using the open-source tool DeepDRR [11]. The pipeline was chosen as it enables the simulation of metal artifact as well as the transfer to real data with only low degradation of prediction performance [10].…”

Section: Methodsmentioning

confidence: 99%

Learning to Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories

Zaech

Gao

Bier

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Metal artifacts in computed tomography (CT) arise from a mismatch between physics of image formation and idealized assumptions during tomographic reconstruction. These artifacts are particularly strong around metal implants, inhibiting widespread adoption of 3D cone-beam CT (CBCT) despite clear opportunity for intra-operative verification of implant positioning, e. g. in spinal fusion surgery. On synthetic and real data, we demonstrate that much of the artifact can be avoided by acquiring better data for reconstruction in a task-aware and patientspecific manner, and describe the first step towards the envisioned taskaware CBCT protocol. The traditional short-scan CBCT trajectory is planar, with little room for scene-specific adjustment. We extend this trajectory by autonomously adjusting out-of-plane angulation. This enables C-arm source trajectories that are scene-specific in that they avoid acquiring "poor images", characterized by beam hardening, photon starvation, and noise. The recommendation of ideal out-of-plane angulation is performed on-the-fly using a deep convolutional neural network that regresses a detectability-rank derived from imaging physics.

show abstract

“…The out-of-plane interval is discretized in steps of 5 • which leads to 11 values to be predicted from each input image. For training, two different datasets were generated by forward projecting 3D volumes using the open-source physics-based X-ray simulator DeepDRR [27,28]. The resulting digitally reconstructed radiographs (DRRs) were created on a uniform grid with step size 5 • in both ϕ and θ .…”

Section: Network For Detectability Predictionmentioning

confidence: 99%

A learning-based method for online adjustment of C-arm Cone-beam CT source trajectories for artifact avoidance

et al. 2020

Self Cite

View full text Add to dashboard Cite

Purpose During spinal fusion surgery, screws are placed close to critical nerves suggesting the need for highly accurate screw placement. Verifying screw placement on high-quality tomographic imaging is essential. C-arm cone-beam CT (CBCT) provides intraoperative 3D tomographic imaging which would allow for immediate verification and, if needed, revision. However, the reconstruction quality attainable with commercial CBCT devices is insufficient, predominantly due to severe metal artifacts in the presence of pedicle screws. These artifacts arise from a mismatch between the true physics of image formation and an idealized model thereof assumed during reconstruction. Prospectively acquiring views onto anatomy that are least affected by this mismatch can, therefore, improve reconstruction quality. Methods We propose to adjust the C-arm CBCT source trajectory during the scan to optimize reconstruction quality with respect to a certain task, i.e., verification of screw placement. Adjustments are performed on-the-fly using a convolutional neural network that regresses a quality index over all possible next views given the current X-ray image. Adjusting the CBCT trajectory to acquire the recommended views results in non-circular source orbits that avoid poor images, and thus, data inconsistencies. Results We demonstrate that convolutional neural networks trained on realistically simulated data are capable of predicting quality metrics that enable scene-specific adjustments of the CBCT source trajectory. Using both realistically simulated data as well as real CBCT acquisitions of a semianthropomorphic phantom, we show that tomographic reconstructions of the resulting scene-specific CBCT acquisitions exhibit improved image quality particularly in terms of metal artifacts. Conclusion The proposed method is a step toward online patient-specific C-arm CBCT source trajectories that enable high-quality tomographic imaging in the operating room. Since the optimization objective is implicitly encoded in a neural network trained on large amounts of well-annotated projection images, the proposed approach overcomes the need for 3D information at run-time.

show abstract

DeepDRR – A Catalyst for Machine Learning in Fluoroscopy-Guided Procedures

Cited by 90 publications

References 15 publications

A gentle introduction to deep learning in medical image processing

A gentle introduction to deep learning in medical image processing

Learning to Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories

A learning-based method for online adjustment of C-arm Cone-beam CT source trajectories for artifact avoidance

Contact Info

Product

Resources

About