RealDRR – Rendering of realistic digitally reconstructed radiographs using locally trained image-to-image translation

Dhont, Jennifer; Verellen, Dirk; Mollaert, Isabelle; Vanreusel, Verdi; Vandemeulebroucke, Jef

doi:10.1016/j.radonc.2020.10.004

Cited by 21 publications

(15 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, although ref. 67 showed that image-to-image translation may closer approximate real X-rays according to image similarity metrics, our study shows that the advantage over domain randomization in terms of downstream task performance is marginal. Finally, because real domain data are being used in both domain adaptation paradigms, adjustments to the real-data target domain, for example, use of a different C-arm X-ray imaging device or design changes to surgical hardware, may require Article https://doi.org/10.1038/s42256-023-00629-1 de novo acquisition of real data and re-training of the models.…”

Section: Discussionmentioning

confidence: 76%

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

Gao

Killeen

et al. 2023

Nat Mach Intell

View full text Add to dashboard Cite

Artificial intelligence (AI) now enables automated interpretation of medical images. However, AI's potential use for interventional image analysis remains largely untapped. This is because the post hoc analysis of data collected during live procedures has fundamental and practical limitations, including ethical considerations, expense, scalability, data integrity and a lack of ground truth. Here we demonstrate that creating realistic simulated images from human models is a viable alternative and complement to large-scale in situ data collection. We show that training AI image analysis models on realistically synthesized data, combined with contemporary domain generalization techniques, results in machine learning models that on real data perform comparably to models trained on a precisely matched real data training set. We find that our model transfer paradigm for X-ray image analysis, which we refer to as SyntheX, can even outperform real-data-trained models due to the effectiveness of training on a larger dataset. SyntheX provides an opportunity to markedly accelerate the conception, design and evaluation of X-ray-based intelligent systems. In addition, SyntheX provides the opportunity to test novel instrumentation, design complementary surgical approaches, and envision novel techniques that improve outcomes, save time or mitigate human error, free from the ethical and practical considerations of live human data collection.Advances in robotics and artificial intelligence (AI) are bringing autonomous surgical systems closer to reality. However, developing the AI backbones of such systems currently depends on collecting training data during routine surgeries. This remains one of the largest barriers to widespread use of AI systems in interventional clinical settings, versus triage or diagnostic settings, as the acquisition and annotation of interventional data is time intensive and costly. Furthermore, while this approach can contribute to the automation or streamlining of existing surgical workflows, robotic and autonomous systems promise even more substantial advances: novel and super-human techniques that improve outcomes, save time or mitigate human error. This is perhaps the most exciting frontier of computer-assisted intervention research.Conventional approaches for curating data for AI development (that is, sourcing it retroactively from clinical practice) are insufficient for training AI models that benefit interventions that use novel instrumentation, different access points or more flexible imaging. This is because they are, by definition, incompatible with contemporary clinical practices and such data do not emerge from routine care. Furthermore, these novel systems are not readily approved, and thus not easily or quickly introduced into clinical practice. Ex vivo experimentation

show abstract

Section: Discussionmentioning

confidence: 76%

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

Gao

Killeen

et al. 2023

Nat Mach Intell

View full text Add to dashboard Cite

show abstract

Section: Two-dimensional Imaging-based Localizationmentioning

confidence: 99%

“…Regarding to the morphologic difference between the real kV projection images and the DRRs, instead of using classical image processing algorithms, a GAN-type framework is proposed by Dhont et al (75) to synthesize DRR from the input kV image acquired from the OBI. This synthetic DRR is able to further improve the accuracy of the aforementioned markerless target positioning method (75). The deep learning-based markerless IGRT methods can not only be applied to the OBI for image guidance, but also to orthogonal kV live images acquired from stereotactic radiosurgery for real-time image guidance (21).…”

Section: Two-dimensional Imaging-based Localizationmentioning

confidence: 99%

Artificial intelligence in image-guided radiotherapy: a review of treatment target localization

Zhao¹,

Shen²,

Islam³

et al. 2021

Quant Imaging Med Surg

View full text Add to dashboard Cite

Modern conformal beam delivery techniques require image-guidance to ensure the prescribed dose to be delivered as planned. Recent advances in artificial intelligence (AI) have greatly augmented our ability to accurately localize the treatment target while sparing the normal tissues. In this paper, we review the applications of AI-based algorithms in image-guided radiotherapy (IGRT), and discuss the indications of these applications to the future of clinical practice of radiotherapy. The benefits, limitations and some important trends in research and development of the AI-based IGRT techniques are also discussed. AI-based IGRT techniques have the potential to monitor tumor motion, reduce treatment uncertainty and improve treatment precision. Particularly, these techniques also allow more healthy tissue to be spared while keeping tumor coverage the same or even better.

show abstract

“…Surface scanning and gated/ tracking techniques open the door to motion management for those limited cases that might benefit from these approaches (eg prostate SBRT, NSCLC and oligometastatic disease). New developments in machine and deep learning (ML/DL) [12] will soon enable markerless tracking and make complicated hybrid developments unnecessary. The argument on superior image quality is subject of an entirely different debate.…”

Section: Present and Future Of Mr Linac-debate Critical Appraisal "Inmentioning

confidence: 99%

X-change symposium: status and future of modern radiation oncology—from technology to biology

et al. 2021

Self Cite

View full text Add to dashboard Cite

Future radiation oncology encompasses a broad spectrum of topics ranging from modern clinical trial design to treatment and imaging technology and biology. In more detail, the application of hybrid MRI devices in modern image-guided radiotherapy; the emerging field of radiomics; the role of molecular imaging using positron emission tomography and its integration into clinical routine; radiation biology with its future perspectives, the role of molecular signatures in prognostic modelling; as well as special treatment modalities such as brachytherapy or proton beam therapy are areas of rapid development. More clinically, radiation oncology will certainly find an important role in the management of oligometastasis. The treatment spectrum will also be widened by the rational integration of modern systemic targeted or immune therapies into multimodal treatment strategies. All these developments will require a concise rethinking of clinical trial design. This article reviews the current status and the potential developments in the field of radiation oncology as discussed by a panel of European and international experts sharing their vision during the “X-Change” symposium, held in July 2019 in Munich (Germany).

show abstract

RealDRR – Rendering of realistic digitally reconstructed radiographs using locally trained image-to-image translation

Cited by 21 publications

References 50 publications

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

Synthetic data accelerates the development of generalizable learning-based algorithms for X-ray image analysis

Artificial intelligence in image-guided radiotherapy: a review of treatment target localization

X-change symposium: status and future of modern radiation oncology—from technology to biology

Contact Info

Product

Resources

About