Dealing with Lack of Training Data for Convolutional Neural Networks: The Case of Digital Pathology

Abstract

Thanks to their capability to learn generalizable descriptors directly from images, deep Convolutional Neural Networks (CNNs) seem the ideal solution to most pattern recognition problems. On the other hand, to learn the image representation, CNNs need huge sets of annotated samples that are unfeasible in many every-day scenarios. This is the case, for example, of Computer-Aided Diagnosis (CAD) systems for digital pathology, where additional challenges are posed by the high variability of the cancerous tissue characteristics. In our experiments, state-of-the-art CNNs trained from scratch on histological images were less accurate and less robust to variability than a traditional machine learning framework, highlighting all the issues of fully training deep networks with limited data from real patients. To solve this problem, we designed and compared three transfer learning frameworks, leveraging CNNs pre-trained on non-medical images. This approach obtained very high accuracy, requiring much less computational resource for the training. Our findings demonstrate that transfer learning is a solution to the automated classification of histological samples and solves the problem of designing accurate and computationally-efficient CAD systems with limited training data.

Multi-Task Pre-Training of Deep Neural Networks for Digital Pathology

“…The success of this approach is mostly due to the possibility to use the large and versatile ImageNet [16] dataset as a source task [26]. Medical imaging and digital pathology communities have therefore studied and used transfer learning [9], [27]- [30] as it provides a way of coping with data scarcity. Those works have explored and evaluated different transfer 2 https://github.com/waliens/multitask-dipath techniques mostly using ImageNet as a source task.…”

Deep learning for colon cancer histopathological images analysis

“…In other words, transfer the already acquired knowledge from different datasets to the WSIs context. This procedure appears in many state-of-the-art methods [43,44,45].…”

Artificial Intelligence & Tissue Biomarkers: Advantages, Risks and Perspectives for Pathology

“…Most of the available AI models have been trained on small data sets and can present a 20% drop of performance when applied in a setting different from where they had been originated. Dataset can be enlarged using specific technical solution such as the transfer learning approach [ 4 , 5 ]. Another possibility will be the development of open sources datasets such as those already hosted by the Cancer Genome Atlas, the Cancer Imaging Archives and Grand Challenges.…”

“…As regard to the dimension of the dataset it is interesting to observe that convolutional neural network (CNN) can be trained on a source task and then be reused on a different target task. This technique, known as transfer learning, can be extremely useful when the data for the target task is scarce but a larger dataset is available to train the source task [ 4 , 5 ]. Finally, in unsupervised DL, the learning examples are provided with no associated labels.…”