Biomarker-Based Classification and Localization of Renal Lesions Using Learned Representations of Histology—A Machine Learning Approach to Histopathology

Abstract

Several deep learning approaches have been proposed to address the challenges in computational pathology by learning structural details in an unbiased way. Transfer learning allows starting from a learned representation of a pretrained model to be directly used or fine-tuned for a new domain. However, in histopathology, the problem domain is tissue-specific and putting together a labelled data set is challenging. On the other hand, whole slide-level annotations, such as biomarker levels, are much easier to obtain. We compare two pretrained models, one histology-specific and one from ImageNet on various computational pathology tasks. We show that a domain-specific model (HistoNet) contains richer information for biomarker classification, localization of biomarker-relevant morphology within a slide, and the prediction of expert-graded features. We use a weakly supervised approach to discriminate slides based on biomarker level and simultaneously predict which regions contribute to that prediction. We employ multitask learning to show that learned representations correlate with morphological features graded by expert pathologists. All of these results are demonstrated in the context of renal toxicity in a mechanistic study of compound toxicity in rat models. Our results emphasize the importance of histology-specific models and their knowledge representations for solving a wide range of computational pathology tasks.

HistoNet: A Deep Learning-Based Model of Normal Histology

“…In addition to being beneficial in the close-at-hand task of cross-species tissue classification, the approach of transfer learning from a domain-specific model described here may also be a suitable starting point when training for other tasks in the histology domain, for example, the recognition of a specific type of lesion for which only limited training data are available. In fact, this hypothesis is validated in another article contained in this special issue 22 that shows, in a study on renal lesions, that the HistoNet pretrained model contains richer information for tasks such as biomarker classification, localization of biomarker-relevant morphology within a slide, as well as the prediction of expert-graded features compared to a model pretrained with ImageNet weights. In this article, Resnet-50 was used as the backbone of the model instead of Inception-v3.…”

“…Therefore, the Inception-v3-based HistoNet models were used as the basis for subsequent analyses, whereas the ResNet-50 variants of HistoNet were applied to specific tasks described elsewhere. 22 Given that the models were intentionally trained to differentiate morphologically similar tissues such as the different segments of the intestine, the absolute accuracy of tissue prediction may not fully reflect the learning of histologically relevant high-level features. To explore this concept, the top-N accuracy was computed for models based on the Inception-v3 architecture.…”

“…Therefore, although based on normal histology, our models could be applicable beyond the task it was defined for, such as in lesion detection or in content-based image retrieval. 22…”

Deep Learning Approaches and Applications in Toxicologic Histopathology: Current Status and Future Perspectives

“…The following tabulated information suggests that these nonclinical applications are on supervised DL (with the exception of Freyre et al . )[ 71 ] in specific species/tissue/abnormality, and attention to DL applications in toxicologic pathology has increased over the past year. The following subsections detail four different categories of DL applications in toxicological pathology: Computer-assisted QC, research-driven computational image analysis, computer-assisted abnormality detection, and content-based image retrieval.…”

Special Issue on Digital Pathology, Tissue Image Analysis, Artificial Intelligence, and Machine Learning: Approximation of the Effect of Novel Technologies on Toxicologic Pathology

“…Hoefling et al describe the development of a deep learning-based model trained on normal histology slides from toxicologic pathology studies. 10 The application of this model to then distinguish normal from abnormal tissue is demonstrated by Freyre et al 11 We believe that toxicologic pathology will benefit from such foundational models which can be adapted for specific purposes or turned for example into general abnormality detectors, rather than having an exploding number of unrelated task-specific models. Kuklyte et al demonstrate the need to consider and the value of multimagnification convolutional neural networks for the determination and quantitation of lesions in nonclinical pathology studies.…”