Iteratively-Refined Interactive 3D Medical Image Segmentation With Multi-Agent Reinforcement Learning

Liao, Xiangyun; Li, Wenhao; Xu, Qisen; Wang, Xiangfeng; Jin, Bo; Zhang, Xiaoyun; Wang, Yanfeng; Zhang, Ya

doi:10.1109/cvpr42600.2020.00941

Cited by 73 publications

(42 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL architectures have also been applied in analyzing medical images obtained from magnetic resonance imaging (MRI), computerized tomography (CT) scan, ultrasound (UlS), etc. However, to the best of our knowledge, the research on medical image segmentation is limited [ 48 , 49 , 50 , 51 , 52 , 53 , 54 ], especially in LV segmentation [ 55 , 56 , 57 ]. In [ 58 ], Mahmud et al reviewed various important applications of deep learning and reinforcement learning to biological data.…”

Section: Related Workmentioning

confidence: 99%

Edge-Sensitive Left Ventricle Segmentation Using Deep Reinforcement Learning

Xiong

Cheung

et al. 2021

Sensors

View full text Add to dashboard Cite

Deep reinforcement learning (DRL) has been utilized in numerous computer vision tasks, such as object detection, autonomous driving, etc. However, relatively few DRL methods have been proposed in the area of image segmentation, particularly in left ventricle segmentation. Reinforcement learning-based methods in earlier works often rely on learning proper thresholds to perform segmentation, and the segmentation results are inaccurate due to the sensitivity of the threshold. To tackle this problem, a novel DRL agent is designed to imitate the human process to perform LV segmentation. For this purpose, we formulate the segmentation problem as a Markov decision process and innovatively optimize it through DRL. The proposed DRL agent consists of two neural networks, i.e., First-P-Net and Next-P-Net. The First-P-Net locates the initial edge point, and the Next-P-Net locates the remaining edge points successively and ultimately obtains a closed segmentation result. The experimental results show that the proposed model has outperformed the previous reinforcement learning methods and achieved comparable performances compared with deep learning baselines on two widely used LV endocardium segmentation datasets, namely Automated Cardiac Diagnosis Challenge (ACDC) 2017 dataset, and Sunnybrook 2009 dataset. Moreover, the proposed model achieves higher F-measure accuracy compared with deep learning methods when training with a very limited number of samples.

show abstract

Section: Related Workmentioning

confidence: 99%

Edge-Sensitive Left Ventricle Segmentation Using Deep Reinforcement Learning

Xiong

Cheung

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…To solve the above issues, this paper models the interactive NE annotation of a whole document as a Markov decision process (MDP), and proposes RL-DINEA, short for Reinforcement Learning-based Document-level Interactive Named Entity Annotation. Reinforcement learning is suitable for interactive annotation tasks due to its intrinsic sequentiality [15]. Document-level label propagation and instance selection are viewed as actions in the MDP.…”

Section: Approachmentioning

confidence: 99%

“…It is well known for its success in the area of game, such as Atari [16] and Go games [25]. RL is suitable for interactive annotation tasks due to its intrinsic sequentiality, and has been applied to interactive annotation of images and videos [15], [26], [27].…”

Section: Reinforcement Learning For Interactive Annotationmentioning

confidence: 99%

“…Liao et al [15] model interactive image segmentation as a MDP and solve it with multi-agent RL. At each step, an RL model predicts the labels of all voxels, based on the previous predictions and human supervisions.…”

Section: Reinforcement Learning For Interactive Annotationmentioning

confidence: 99%

“…Our RL-DINEA differs from previous RL-based interactive annotation approaches in two ways. First, previous approaches adopt RL either to learn prediction [15], or to learn instance selection [26], [27], while RL-DINEA jointly learns prediction and instance selection. Second, previous approaches apply RL in image and video annotation tasks, while RL-DINEA is for named entity annotation.…”

Section: Reinforcement Learning For Interactive Annotationmentioning

confidence: 99%

See 2 more Smart Citations

Learning Document-Level Label Propagation and Instance Selection by Deep Q-Network for Interactive Named Entity Annotation

Gui

Gao

2021

IEEE Access

View full text Add to dashboard Cite

High quality annotated named entity corpora are essential language resources, but fully manual annotation is time-consuming. Interactive annotation offers an efficient alternative where humans and machines collaborate. Instances of named entity mentions tend to share the same label, when they cooccur in the same document and have similar surface forms. After selecting an instance in one sentence for manual annotation, the label of the instance can be propagated to instances in other sentences. This kind of document-level label propagation can be used to reduce human effort and improve annotation quality in interactive annotation. However, most existing literature assumes instances within different sentences are independent, and ignores document-level label propagation. This paper proposes a reinforcement learningbased approach, which learns to propagate labels among the instances within a document for interactive named entity annotation. In addition, our approach also learns instance selection for manual annotation. We optimize the objective which is a trade-off between human effort and annotation quality by training a deep Q-network. Our approach reduces human effort by more than 42% compared to baseline approaches, for achieving the same annotation quality (0.95 measured by F1 averaged on three datasets).

show abstract

Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions

Zhang

Luke

et al. 2023

J Applied Clin Med Phys

View full text Add to dashboard Cite

Motivation Medical image analysis involves a series of tasks used to assist physicians in qualitative and quantitative analyses of lesions or anatomical structures which can significantly improve the accuracy and reliability of medical diagnoses and prognoses. Traditionally, these tedious tasks were finished by experienced physicians or medical physicists and were marred with two major problems, low efficiency and bias. In the past decade, many machine learning methods have been applied to accelerate and automate the image analysis process. Compared to the enormous deployments of supervised and unsupervised learning models, attempts to use reinforcement learning in medical image analysis are still scarce. We hope that this review article could serve as the stepping stone for related research in the future. Significance We found that although reinforcement learning has gradually gained momentum in recent years, many researchers in the medical analysis field still find it hard to understand and deploy in clinical settings. One possible cause is a lack of well‐organized review articles intended for readers without professional computer science backgrounds. Rather than to provide a comprehensive list of all reinforcement learning models applied in medical image analysis, the aim of this review is to help the readers formulate and solve their medical image analysis research through the lens of reinforcement learning. Approach & Results We selected published articles from Google Scholar and PubMed. Considering the scarcity of related articles, we also included some outstanding newest preprints. The papers were carefully reviewed and categorized according to the type of image analysis task. In this article, we first reviewed the basic concepts and popular models of reinforcement learning. Then, we explored the applications of reinforcement learning models in medical image analysis. Finally, we concluded the article by discussing the reviewed reinforcement learning approaches’ limitations and possible future improvements.

show abstract

Iteratively-Refined Interactive 3D Medical Image Segmentation With Multi-Agent Reinforcement Learning

Cited by 73 publications

References 14 publications

Edge-Sensitive Left Ventricle Segmentation Using Deep Reinforcement Learning

Edge-Sensitive Left Ventricle Segmentation Using Deep Reinforcement Learning

Learning Document-Level Label Propagation and Instance Selection by Deep Q-Network for Interactive Named Entity Annotation

Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions

Contact Info

Product

Resources

About