Evaluation of a Reinforcement Learning Algorithm for Vascular Intervention Surgery

Meng, Fanxu; Guo, Shuxiang; Zhou, Wei; Chen, Zhengyang

doi:10.1109/icma52036.2021.9512675

Cited by 6 publications

(24 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As shown in Figure 1, 462 studies met the search criteria, and 21 full-text studies were assessed against the eligibility criteria. A total of 14 were identified for review (Rafii-Tari et al, 2013Chi et al, 2018aChi et al, ,b, 2020Behr et al, 2019;You et al, 2019;Zhao et al, 2019;Kweon et al, 2021;Meng et al, 2021Meng et al, , 2022Cho et al, 2022;Karstensen et al, 2022;Wang et al, 2022). The characteristics of the fourteen studies are listed in Table 2.…”

Section: Studiesmentioning

confidence: 99%

“…. RL methods RL was used in nine studies (9/14, 64%) with algorithms including A3C, DDPG, DQN, Dueling DQN, HER, PI 2 , PPO, and Rainbow (Chi et al, 2018a(Chi et al, , 2020Behr et al, 2019;You et al, 2019;Kweon et al, 2021;Meng et al, 2021Meng et al, , 2022Cho et al, 2022;Karstensen et al, 2022). Demonstrator data in some form (GAIL, Behavior Cloning, or HD) was used as a precursor in four of the studies (4/14, 29%) during training (LfD), in conjunction with other RL algorithms (Chi et al, 2018a;Behr et al, 2019;Kweon et al, 2021;Cho et al, 2022).…”

Section: Yolo Supervised Learningmentioning

confidence: 99%

“…Demonstrator data in some form (GAIL, Behavior Cloning, or HD) was used as a precursor in four of the studies (4/14, 29%) during training (LfD), in conjunction with other RL algorithms (Chi et al, 2018a;Behr et al, 2019;Kweon et al, 2021;Cho et al, 2022). The SOFA framework (Inria, Strasbourg, France; Faure et al, 2012) was used for training RL models in four studies (4/14, 29%; Behr et al, 2019;Cho et al, 2022;Karstensen et al, 2022;Meng et al, 2022), the Unity engine (Unity Technologies, San Francisco, USA) was used in two studies (2/14, 14%; You et al, 2019;Meng et al, 2021), while the platform used for training was not specified in three…”

Section: Yolo Supervised Learningmentioning

confidence: 99%

See 2 more Smart Citations

Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review

Robertshaw

Karstensen

Jackson

et al. 2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

BackgroundAutonomous navigation of catheters and guidewires in endovascular interventional surgery can decrease operation times, improve decision-making during surgery, and reduce operator radiation exposure while increasing access to treatment.ObjectiveTo determine from recent literature, through a systematic review, the impact, challenges, and opportunities artificial intelligence (AI) has for the autonomous navigation of catheters and guidewires for endovascular interventions.MethodsPubMed and IEEEXplore databases were searched to identify reports of AI applied to autonomous navigation methods in endovascular interventional surgery. Eligibility criteria included studies investigating the use of AI in enabling the autonomous navigation of catheters/guidewires in endovascular interventions. Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), articles were assessed using Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). PROSPERO: CRD42023392259.ResultsFour hundred and sixty-two studies fulfilled the search criteria, of which 14 studies were included for analysis. Reinforcement learning (RL) (9/14, 64%) and learning from expert demonstration (7/14, 50%) were used as data-driven models for autonomous navigation. These studies evaluated models on physical phantoms (10/14, 71%) and in-silico (4/14, 29%) models. Experiments within or around the blood vessels of the heart were reported by the majority of studies (10/14, 71%), while non-anatomical vessel platforms “idealized” for simple navigation were used in three studies (3/14, 21%), and the porcine liver venous system in one study. We observed that risk of bias and poor generalizability were present across studies. No procedures were performed on patients in any of the studies reviewed. Moreover, all studies were limited due to the lack of patient selection criteria, reference standards, and reproducibility, which resulted in a low level of evidence for clinical translation.ConclusionDespite the potential benefits of AI applied to autonomous navigation of endovascular interventions, the field is in an experimental proof-of-concept stage, with a technology readiness level of 3. We highlight that reference standards with well-identified performance metrics are crucial to allow for comparisons of data-driven algorithms proposed in the years to come.Systematic review registrationidentifier: CRD42023392259.

show abstract

Section: Studiesmentioning

confidence: 99%

Section: Yolo Supervised Learningmentioning

confidence: 99%

Section: Yolo Supervised Learningmentioning

confidence: 99%

See 1 more Smart Citation

Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review

Robertshaw

Karstensen

Jackson

et al. 2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Analogous to physicians using X-ray fluoroscopy for intraoperative navigation [28], it is more realistic to use images for instrument-manipulation skills. Several state-of-the-art RL algorithms, such as Dueling Deep Q-Network [29] and Asynchronous Advantage Actor-Critic [30], have been applied with preoperative vascular models to learn instrument-manipulation skills with high-dimension images [31,32]. Due to notorious sample inefficiency of RL, existing research is limited to digital simulation environments [31,32].…”

Section: Introductionmentioning

confidence: 99%

“…Several state-of-the-art RL algorithms, such as Dueling Deep Q-Network [29] and Asynchronous Advantage Actor-Critic [30], have been applied with preoperative vascular models to learn instrument-manipulation skills with high-dimension images [31,32]. Due to notorious sample inefficiency of RL, existing research is limited to digital simulation environments [31,32]. However, there is a non-negligible gap between digital simulations and real environments, which limits the clinical value of learned instrument-manipulation skills.…”

Section: Introductionmentioning

confidence: 99%

Discrete soft actor-critic with auto-encoder on vascular robotic system

Zhou

Xie

et al. 2022

Robotica

View full text Add to dashboard Cite

Instrument delivery is critical part in vascular intervention surgery. Due to the soft-body structure of instruments, the relationship between manipulation commands and instrument motion is non-linear, making instrument delivery challenging and time-consuming. Reinforcement learning has the potential to learn manipulation skills and automate instrument delivery with enhanced success rates and reduced workload of physicians. However, due to the sample inefficiency when using high-dimensional images, existing reinforcement learning algorithms are limited on realistic vascular robotic systems. To alleviate this problem, this paper proposes discrete soft actor-critic with auto-encoder (DSAC-AE) that augments SAC-discrete with an auxiliary reconstruction task. The algorithm is applied with distributed sample collection and parameter update in a robot-assisted preclinical environment. Experimental results indicate that guidewire delivery can be automatically implemented after 50k sampling steps in less than 15 h, demonstrating the proposed algorithm has the great potential to learn manipulation skill for vascular robotic systems.

show abstract

Towards an optimization of catheter guidance in vascular surgery: A comparative analysis of the contribution of reinforcement learning

Bouden

2024

Intelligent Surgery

View full text Add to dashboard Cite

Evaluation of a Reinforcement Learning Algorithm for Vascular Intervention Surgery

Cited by 6 publications

References 21 publications

Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review

Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review

Discrete soft actor-critic with auto-encoder on vascular robotic system

Towards an optimization of catheter guidance in vascular surgery: A comparative analysis of the contribution of reinforcement learning

Contact Info

Product

Resources

About