Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions

Abstract

Motivation Medical image analysis involves a series of tasks used to assist physicians in qualitative and quantitative analyses of lesions or anatomical structures which can significantly improve the accuracy and reliability of medical diagnoses and prognoses. Traditionally, these tedious tasks were finished by experienced physicians or medical physicists and were marred with two major problems, low efficiency and bias. In the past decade, many machine learning methods have been applied to accelerate and automate the image analysis process. Compared to the enormous deployments of supervised and unsupervised learning models, attempts to use reinforcement learning in medical image analysis are still scarce. We hope that this review article could serve as the stepping stone for related research in the future. Significance We found that although reinforcement learning has gradually gained momentum in recent years, many researchers in the medical analysis field still find it hard to understand and deploy in clinical settings. One possible cause is a lack of well‐organized review articles intended for readers without professional computer science backgrounds. Rather than to provide a comprehensive list of all reinforcement learning models applied in medical image analysis, the aim of this review is to help the readers formulate and solve their medical image analysis research through the lens of reinforcement learning. Approach & Results We selected published articles from Google Scholar and PubMed. Considering the scarcity of related articles, we also included some outstanding newest preprints. The papers were carefully reviewed and categorized according to the type of image analysis task. In this article, we first reviewed the basic concepts and popular models of reinforcement learning. Then, we explored the applications of reinforcement learning models in medical image analysis. Finally, we concluded the article by discussing the reviewed reinforcement learning approaches’ limitations and possible future improvements.

Prostate gleason score prediction via MRI using capsule network

“…Radiomics, the extraction of image features, has been used to build models aiding in lesion detection, lesion synthesis and cancer prognosis [9,10]. In recent years, many deep learning models have achieved state-of-the-art results in image diagnosis [11][12][13], organ segmentation [14,15] and treatment planning [16,17] in medicine. Recent advances in deep learning have also demonstrated success of convolutional neural networks (CNN) to detect cancer from MRI.…”

Ultrasound-based dominant intraprostatic lesion classification with Swin Transformer

“…Due to its feasibility in detecting DILs, B-mode ultrasound could be potentially integrated into the intraoperative treatment planning for a focal boost [2]. With the development of computer-aided diagnosis (CAD), powerful artificial intelligence (AI) tools such as deep learning (DL) and reinforcement learning algorithms have been applied in medical image analysis [3][4][5][6][7][8][9][10], such as image segmentation [11], image registration [12] and image synthesis [13]. With deeper layers, DL methods can adaptively extracts feature maps at multiple resolution levels, and demonstrates strength in computer vision [14].…”

Ultrasound breast tumor detection based on vision graph neural network

“…They can automatically extract the image features for precise predictions in various computer vision (CV) tasks [7]. In medical imaging, DL-powered systems have significantly changed the landscape with unprecedented processing speed and accuracy [8][9][10][11][12][13][14]. Currently, convolutional neural networks (CNNs) [15] and Vision Transformers [16] are the most widely used backbone for these frameworks.…”