Optimization of deep learning methods for visualization of tumor heterogeneity and brain tumor grading through digital pathology

Abstract

Background Variations in prognosis and treatment options for gliomas are dependent on tumour grading. When tissue is available for analysis, grade is established based on histological criteria. However, histopathological diagnosis is not always reliable or straight-forward due to tumour heterogeneity, sampling error and subjectivity, and hence there is great inter-observer variability in readings. Methods We trained convolutional neural network models to classify digital whole-slide histopathology images from The Cancer Genome Atlas. We tested a number of optimisation parameters. Results Data augmentation did not improve model training, while smaller batch size helped to prevent overfitting and led to improved model performance. There was no significant difference in performance between a modular 2-class model and a single 3-class model system. The best models trained achieved a mean accuracy of 73% in classifying glioblastoma from other grades, and 53% between WHO grade II and III gliomas. A visualisation method was developed to convey the model output in a clinically relevant manner by overlaying colour-coded predictions over the original whole slide image. Conclusions Our developed visualisation method reflects the clinical decision-making process by highlighting the intra-tumour heterogeneity and may be used in clinical setting to aid diagnosis. Explainable AI techniques may allow further evaluation of the model and underline areas for improvements such as biases. Due to intra-tumour heterogeneity, data annotation for training was imprecise, and hence performance was lower than expected. The models may be further improved by employing advanced data augmentation strategies and using more precise semi-automatic or manually labelled training data.

Classification of Diffuse Glioma Subtype from Clinical-Grade Pathological Images Using Deep Transfer Learning

“…The possibility of the dataset without annotation allows our algorithm to learn from the full files of slides that are presented to clinicians from real-world clinical practice, representing the full wealth of biological and technical variablitiy [ 34 ]. To the best of our knowledge, only robust studies were conducted with the deep learning approach using the publicly available digital WSI dataset in The Cancer Genome Atlas (TCGA) or The Cancer Imaging Archive (TCIA) to automate the classification of grade II, III glioma versus grade IV glioblastoma, which demonstrated up to 96% accuracy [ 35 , 36 , 37 ] However, the datasets used in the above studies are composed of many cases diagnosed before the application of the WHO’s new 2016 classification; therefore, the algorithm that was developed using the public database might not be suitable for the current WHO classification system. A recent study tried deep learning approaches for subtype classification according to the 2016 WHO classification and survival prediction using multimodal magnetic resonance images of a brain tumor [ 38 ].…”

Mutual Contrastive Low-rank Learning to Disentangle Whole Slide Image Representations for Glioma Grading

“…Modelling WSI remains challenging due to the gigapixel size of the images and the complexity of tumor tissue. To mitigate the challenge of computational costs, multiple instance learning (MIL) is used as an effective method in model training [10,21]. Using MIL, previous studies proposed machine learning approaches based on feature engineering [22,18].…”

“…Recently, deep learning has achieved superior performance using the end-to-end training scheme. Most state-of-theart models employed the transfer learning approach to transfer the pre-trained weights from ImageNet [10,6,19,21]. Based on the FFPE sections, these studies might be affected by the bias from FFPE tissue.…”

CAD systems for colorectal cancer from WSI are still not ready for clinical acceptance

“…In particular, convolutional neural networks (CNN) are applied to images and automatically extract features, which are then used to identify objects/regions of interest or to classify the underlying diagnosis 26 . In digital pathology, this type of models is used, for example, for mitosis detection 27 , 28 , tissue segmentation 29 , 30 , cancer grading 31 , 32 or histological classification 33 , 34 .…”