Deep reinforcement learning with automated label extraction from clinical reports accurately classifies 3D MRI brain volumes

Abstract

Purpose Image classification is perhaps the most fundamental task in imaging artificial intelligence. However, labeling images is time-consuming and tedious. We have recently demonstrated that reinforcement learning (RL) can classify 2D slices of MRI brain images with high accuracy.Here we make two important steps toward speeding image classification: Firstly, we automatically extract class labels from the clinical reports. Secondly, we extend our prior 2D classification work to fully 3D image volumes from our institution. Hence, we proceed as follows: in Part 1, we extract labels from reports automatically using a natural language processing approach termed sentence bidirectional encoder representations from transformers (SBERT). Then, in Part 2, we use these labels with RL to train a classification Deep-Q Network (DQN) for 3D image volumes.Materials and Methods For Part 1, we trained SBERT with 45 "normal" patient report impressions and 45 metastasis-containing impressions. We then used the trained SBERT to predict class labels for use in Part 2. In Part 2, we applied multi-step image classification to allow for combined Deep-Q learning using 3D convolutions and TD(0) Q learning. We trained on a set of 90 images (40 normal and 50 tumor-containing). We tested on a separate set of 61 images (40 normal and 21 tumor-containing), again using the classes predicted from patient reports by the trained SBERT in Part 1. For comparison, we also trained and tested a supervised deep learning classification network on the same set of training and testing images using the same labels.Results Part 1: Upon training with the corpus of radiology reports, the SBERT model had 100% accuracy for both normal and metastasiscontaining scans. Part 2: Then, using these labels, whereas the supervised

Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions

“…To overcome this limitation, Stember & Shalu published two papers 30,55 which used a DQN agent in tandem with a TD model for accurate image classification with a minimal training set.The main workflows of these two papers were identical, except that the labels in the second paper were extracted from clinical reports using an SBERT. 56 By overlaying the images with red or green masks, they formulated the classification problem as a behavioral problem, shown in Figure 13.…”

Deep Neuroevolution Squeezes More out of Small Neural Networks and Small Training Sets: Sample Application to MRI Brain Sequence Classification

“…We train the DQN weights via SGD. To sample from the image environment, we perform DQN training in tandem with temporal difference learning in an essentially identical manner to that of recent DRL-based binary classification work [18,17].…”

“…As such, DRL may help to lead us past the above-described impasse in Radiology AI. In fact, early applications of DRL have shown remarkable ability to generalize based on small training sets [16,20,19,17,18].…”