Targeted transfer learning to improve performance in small medical physics datasets

Abstract

Purpose To perform an in‐depth evaluation of current state of the art techniques in training neural networks to identify appropriate approaches in small datasets. Method In total, 112,120 frontal‐view X‐ray images from the NIH ChestXray14 dataset were used in our analysis. Two tasks were studied: unbalanced multi‐label classification of 14 diseases, and binary classification of pneumonia vs non‐pneumonia. All datasets were randomly split into training, validation, and testing (70%, 10%, and 20%). Two popular convolution neural networks (CNNs), DensNet121 and ResNet50, were trained using PyTorch. We performed several experiments to test: (a) whether transfer learning using pretrained networks on ImageNet are of value to medical imaging/physics tasks (e.g., predicting toxicity from radiographic images after training on images from the internet), (b) whether using pretrained networks trained on problems that are similar to the target task helps transfer learning (e.g., using X‐ray pretrained networks for X‐ray target tasks), (c) whether freeze deep layers or change all weights provides an optimal transfer learning strategy, (d) the best strategy for the learning rate policy, and (e) what quantity of data is needed in order to appropriately deploy these various strategies (N = 50 to N = 77 880). Results In the multi‐label problem, DensNet121 needed at least 1600 patients to be comparable to, and 10 000 to outperform, radiomics‐based logistic regression. In classifying pneumonia vs non‐pneumonia, both CNN and radiomics‐based methods performed poorly when N < 2000. For small datasets ( < 2000), however, a significant boost in performance (>15% increase on AUC) comes from a good selection of the transfer learning dataset, dropout, cycling learning rate, and freezing and unfreezing of deep layers as training progresses. In contrast, if sufficient data are available (>35 000), little or no tweaking is needed to obtain impressive performance. While transfer learning using X‐ray images from other anatomical sites improves performance, we also observed a similar boost by using pretrained networks from ImageNet. Having source images from the same anatomical site, however, outperforms every other methodology, by up to 15%. In this case, DL models can be trained with as little as N = 50. Conclusions While training DL models in small datasets (N < 2000) is challenging, no tweaking is necessary for bigger datasets (N > 35 000). Using transfer learning with images from the same anatomical site can yield remarkable performance in new tasks with as few as N = 50. Surprisingly, we did not find any advantage for using images from other anatomical sites over networks that have been trained using ImageNet. This indicates that features learned may not be as general as currently believed, and performance decays rapidly even by just changing the anatomical site of the images.

Deep Learning Segmentation of Glomeruli on Kidney Donor Frozen Sections

“…Our results demonstrate that the relative effect of transfer learning is indirectly proportional to sample size. This finding is consistent with machine learning theory 62,63 and implies that the performance of our sclerotic glomeruli model may asymptotically improve with more data. Second, data augmentation was also shown to increase the generalization of our DL results, suggesting artificial augmentations may capture important characteristics of renal pathology 64 .…”

Segmentation and classification on chest radiography: a systematic survey

“…AG-Mask Interface 87.1 DL [ 136 ] lr = 0.0001 Adam Softmax 224 224 Images are resized using the bilinear interpolation and are also horizontally flipped CXR-14 ResNet-18, Transfer learning, CAM, Data augmentation 84.94 DL [ 126 ] lr = 0.00005 SGD Sigmoid 1024 1024 Images are resized using the bilinear interpolation and additional techniques such as rotation, zoom and shifting are performed for data augmentation AMC, SNUBH ResNet-50, fine-tuning, CAM, curriculum learning 98.3 DL [ 178 ] lr = 0.0001 Adam ReLu, Sigmoid 128 128 Images are normalized to enhance the contrast. Further, rotation, scaling and flipping is performed for reducing the overfitting CXR-14 Inception-ResNet-v2, dilated ResNet, transfer Learning, cross-validation 90.0 DL [ 44 ] lr = 0.0001 and it is decreased by 10 times after every 3 epochs Adam Softmax, Sigmoid 224 224 Images normalization is performed CXR-14 ResNet-18, DenseNet-121, MSML 84.32 DL [ 140 ] – – – 236 236 Images are resized, cropped, flipped and rotated …”

“…All the pre-trained models are generally trained on the ImageNet dataset. Romero et al [ 140 ] have reviewed, evaluated, and compared the state-of-the-art training techniques on the small datasets. Deep CNN model performance has been checked on small datasets for emphysema detection, pneumonia detection, hernia detection, and CXR-14 classification with or without transfer learning.…”

Application of Artificial Intelligence in Diagnosis of Craniopharyngioma

“…Considering that most of tasks are correlated, transfer learning can accelerate and optimize the learning efficiency of the model by transferring the pre-trained model parameters to the new model, rather than learning from scratch. For deep learning models, the transfer method applies the pre-training model to a new task by fine-tuning ( 105 ). For the transfer learning method, the source domain and the target domain do not need the same distribution of data, which overcomes the shortcomings of traditional machine learning, and has a great advantage in the case of few samples in the target domain and sufficient samples in related fields.…”