Deep reinforcement learning to detect brain lesions on MRI: a proof-of-concept application of reinforcement learning to medical images

Stember, Joseph N.; Shalu, Hrithwik

doi:10.48550/arxiv.2008.02708

Cited by 13 publications

(29 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to predict the actions our agent will take, we use the Deep-Q Network (DQN), as we have in prior work [9,10,11,12]. The architecture of our DQN is illustrated in Figure 1.…”

Section: Deep-q Networkmentioning

confidence: 99%

“…We should note at this point that selecting argmax a (Q) is an "on-policy" action selection. As we will see, and have described in prior work [9,10,11,12], we also need to experiment with random action selections to explore and learn about the environment. This is called off-policy behavior.…”

Section: Deep-q Networkmentioning

confidence: 99%

“…As the agent learns about the environment and the best policy to follow, the degree of randomness decreases and the agent increasingly chooses the optimal action predicted by the DQN, i.e., the argmax a {Q(a = 0), Q(a = 1)}. This is called the epsilon-greedy algorithm, and it is described in more detail in our prior RL works [9,10,11,12]. We use similar parameters regarding this part of the approach:…”

Section: Temporal Difference Learningmentioning

confidence: 99%

“…We can store all of these values in a tuple called a transition. As described in our prior RL work [9,10,11,12], we store these up to a maximum memory size called the replay memory buffer. Here, in order to conserve space and be able to save more transitions in the replay memory buffer, we store state information solely based on the training image number and correctness of the prior and just-made predictions, pred_corr t−1 and pred_corr t , respectively.…”

Section: Temporal Difference Learningmentioning

confidence: 99%

See 3 more Smart Citations

Deep reinforcement learning with automated label extraction from clinical reports accurately classifies 3D MRI brain volumes

Stember¹,

Shalu²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Purpose Image classification is perhaps the most fundamental task in imaging artificial intelligence. However, labeling images is time-consuming and tedious. We have recently demonstrated that reinforcement learning (RL) can classify 2D slices of MRI brain images with high accuracy.Here we make two important steps toward speeding image classification: Firstly, we automatically extract class labels from the clinical reports. Secondly, we extend our prior 2D classification work to fully 3D image volumes from our institution. Hence, we proceed as follows: in Part 1, we extract labels from reports automatically using a natural language processing approach termed sentence bidirectional encoder representations from transformers (SBERT). Then, in Part 2, we use these labels with RL to train a classification Deep-Q Network (DQN) for 3D image volumes.Materials and Methods For Part 1, we trained SBERT with 45 "normal" patient report impressions and 45 metastasis-containing impressions. We then used the trained SBERT to predict class labels for use in Part 2. In Part 2, we applied multi-step image classification to allow for combined Deep-Q learning using 3D convolutions and TD(0) Q learning. We trained on a set of 90 images (40 normal and 50 tumor-containing). We tested on a separate set of 61 images (40 normal and 21 tumor-containing), again using the classes predicted from patient reports by the trained SBERT in Part 1. For comparison, we also trained and tested a supervised deep learning classification network on the same set of training and testing images using the same labels.Results Part 1: Upon training with the corpus of radiology reports, the SBERT model had 100% accuracy for both normal and metastasiscontaining scans. Part 2: Then, using these labels, whereas the supervised

show abstract

“…In order to predict the actions our agent will take, we use the Deep-Q Network (DQN), as we have in prior work [9,10,11,12]. The architecture of our DQN is illustrated in Figure 1.…”

Section: Deep-q Networkmentioning

confidence: 99%

Section: Deep-q Networkmentioning

confidence: 99%

Section: Temporal Difference Learningmentioning

confidence: 99%

Section: Temporal Difference Learningmentioning

confidence: 99%

See 2 more Smart Citations

Deep reinforcement learning with automated label extraction from clinical reports accurately classifies 3D MRI brain volumes

Stember¹,

Shalu²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…As such, DRL may help to lead us past the above-described impasse in Radiology AI. In fact, early applications of DRL have shown remarkable ability to generalize based on small training sets [16,20,19,17,18].…”

Section: Introductionmentioning

confidence: 99%

Deep Neuroevolution Squeezes More out of Small Neural Networks and Small Training Sets: Sample Application to MRI Brain Sequence Classification

Stember¹,

Shalu²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Purpose Deep Neuroevolution (DNE) holds the promise of providing radiology artificial intelligence (AI) that performs well with small neural networks and small training sets. We seek to realize this potential via a proof-of-principle application to MRI brain sequence classification.Materials and Methods We analyzed a training set of 20 patients, each with four sequences/weightings: T1, T1 post-contrast, T2, and T2-FLAIR. We trained the parameters of a relatively small convolutional neural network (CNN) as follows: First, we randomly mutated the CNN weights. We then measured the CNN training set accuracy, using the latter as the fitness evaluation metric. The fittest "child" CNNs were identified. We incorporated their mutations into the "parent" CNN. This selectively mutated parent became the next generation's parent CNN. We repeated this process for approximately 50,000 generations.Results DNE achieved monotonic convergence to 100% training set accuracy. DNE also converged monotonically to 100 % testing set accuracy.Conclusion DNE can achieve perfect accuracy with small training sets and small CNNs. Particularly when combined with Deep Reinforcement Learning, DNE may provide a path forward in the quest to make radiology AI more human-like in its ability to learn. DNE may very well turn out to be a key component of the much-anticipated "meta-learning" regime of radiology AI; algorithms that can adapt to new tasks and new image types, similar to human radiologists.

show abstract

Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions

Zhang

Luke

et al. 2023

J Applied Clin Med Phys

View full text Add to dashboard Cite

Motivation Medical image analysis involves a series of tasks used to assist physicians in qualitative and quantitative analyses of lesions or anatomical structures which can significantly improve the accuracy and reliability of medical diagnoses and prognoses. Traditionally, these tedious tasks were finished by experienced physicians or medical physicists and were marred with two major problems, low efficiency and bias. In the past decade, many machine learning methods have been applied to accelerate and automate the image analysis process. Compared to the enormous deployments of supervised and unsupervised learning models, attempts to use reinforcement learning in medical image analysis are still scarce. We hope that this review article could serve as the stepping stone for related research in the future. Significance We found that although reinforcement learning has gradually gained momentum in recent years, many researchers in the medical analysis field still find it hard to understand and deploy in clinical settings. One possible cause is a lack of well‐organized review articles intended for readers without professional computer science backgrounds. Rather than to provide a comprehensive list of all reinforcement learning models applied in medical image analysis, the aim of this review is to help the readers formulate and solve their medical image analysis research through the lens of reinforcement learning. Approach & Results We selected published articles from Google Scholar and PubMed. Considering the scarcity of related articles, we also included some outstanding newest preprints. The papers were carefully reviewed and categorized according to the type of image analysis task. In this article, we first reviewed the basic concepts and popular models of reinforcement learning. Then, we explored the applications of reinforcement learning models in medical image analysis. Finally, we concluded the article by discussing the reviewed reinforcement learning approaches’ limitations and possible future improvements.

show abstract

Deep reinforcement learning to detect brain lesions on MRI: a proof-of-concept application of reinforcement learning to medical images

Cited by 13 publications

References 0 publications

Deep reinforcement learning with automated label extraction from clinical reports accurately classifies 3D MRI brain volumes

Deep reinforcement learning with automated label extraction from clinical reports accurately classifies 3D MRI brain volumes

Deep Neuroevolution Squeezes More out of Small Neural Networks and Small Training Sets: Sample Application to MRI Brain Sequence Classification

Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions

Contact Info

Product

Resources

About