A Deep Reinforced Tree-Traversal Agent for Coronary Artery Centerline Extraction

Deep reinforcement learning for cerebral anterior vessel tree extraction from 3D CTA images

“…This result can be observed from both DQN methods and PPO methods. The curve-to-curve distance reward aims to optimize the surface area between the reference standard and agent path at 𝑡 and 𝑡 − 1, whereas the point-to-curve distance used by the work of Zhang et al (2018) and Li et al (2021) aim to optimize the distance between the agent position and corresponding curves. Using the curve-to-curve distance reward function, the instant reward at 𝑡 is approximated by the difference in surface area between the agent path at 𝑡 and 𝑡 − 1 with regard to the complete reference standard.…”

“…In this work, the bifurcations were detected using a standalone detector. Li et al (2021) utilize the DQN approach with predefined orientation as discrete action space to track coronary centerlines. In addition, the value-based methods are known to often lead to an overestimation of the action value (Van Hasselt et al, 2016).…”

“…In the ablation study, we investigate the importance of our network architecture design choices, training scheme, and reward function. In addition, we compare our reward function with an existing reward from Li et al (2021). We, therefore, divided our ablation study into two parts.…”

“…• curve-to-curve distance only, by using the log curve-to-curve distance function (CC distance model), • the point to curve distance reward of Li et al (2021) (Li reward) Fig. 5 shows the learning curves with CCS metric and Table 4 contains the test results for the ablation study of network architecture and the training scheme.…”

“…We also compared our method with a popular DQN method. Our baseline model was introduced by Li et al (2021). This method uses the same network architecture as Wolterink et al (2019), which is similar to the CNN architecture of our actor network but with one time point and ReLU activation function.…”