HistoNet: A Deep Learning-Based Model of Normal Histology

Abstract

We introduce HistoNet, a deep neural network trained on normal tissue. On 1690 slides with rat tissue samples from 6 preclinical toxicology studies, tissue regions were outlined and annotated by pathologists into 46 different tissue classes. From these annotated regions, we sampled small 224 × 224 pixels images (patches) at 6 different levels of magnification. Using 4 studies as training set and 2 studies as test set, we trained VGG-16, ResNet-50, and Inception-v3 networks separately at each magnification level. Among these model architectures, Inception-v3 and ResNet-50 outperformed VGG-16. Inception-v3 identified the tissue from query images, with an accuracy up to 83.4%. Most misclassifications occurred between histologically similar tissues. Investigation of the features learned by the model (embedding layer) using Uniform Manifold Approximation and Projection revealed not only coherent clusters associated with the individual tissues but also subclusters corresponding to histologically meaningful structures that had not been annotated or trained for. This suggests that the histological representation learned by HistoNet could be useful as the basis of other machine learning algorithms and data mining. Finally, we found that models trained on rat tissues can be used on non-human primate and minipig tissues with minimal retraining.

Biomarker-Based Classification and Localization of Renal Lesions Using Learned Representations of Histology—A Machine Learning Approach to Histopathology

“…HistoNet 17 is a set of deep neural networks characterizing the diversity of normal tissues that provide rich learned representations that can be extended to wider problems in computational pathology. The networks have been trained on 1,690 slides with rat tissue samples from 6 preclinical toxicology studies where tissue regions were outlined and annotated by pathologists into 46 different tissue classes.…”

“…We used either a ResNet-50 pretrained on the ImageNet database 16 or the ResNet-50 presented in the HistoNet work. 17 The HistoNet ResNet-50 is a ResNet-50 which has been pretrained on a set of histological studies containing only normal rat tissues (see the original work for more information 17 ). The penultimate layer of the ResNet-50 (2048 dimensional) was extracted for both pretrained models (ResNet-50 pretrained ImageNet and ResNet-50 pretrained HistoNet).…”

“…Recently, a novel DL model trained on 46 different tissue types of normal rat histology was reported, which could provide transferable knowledge for many image analysis tasks on histology images. 17 Applying DL methods in digital histopathology has some specific challenges. First, WSIs are much larger (in the gigapixel range) than images found in other domains.…”

“…In this study, we use 2 pretrained CNNs (ResNet-50), the first model was pretrained on the publicly available ImageNet data set 1,16 and the second, named HistoNet, 17 was pretrained on histology slides from several normal rat tissues. These models are employed to extract visual representations from previously unseen histology images and compare their transfer learning performance for lesion classification and localization in the context of mechanistic rat toxicology studies with nephrotoxic compounds.…”

Special Issue on Digital Pathology, Tissue Image Analysis, Artificial Intelligence, and Machine Learning: Approximation of the Effect of Novel Technologies on Toxicologic Pathology

“…Hoefling et al describe the development of a deep learning-based model trained on normal histology slides from toxicologic pathology studies. 10 The application of this model to then distinguish normal from abnormal tissue is demonstrated by Freyre et al 11 We believe that toxicologic pathology will benefit from such foundational models which can be adapted for specific purposes or turned for example into general abnormality detectors, rather than having an exploding number of unrelated task-specific models. Kuklyte et al demonstrate the need to consider and the value of multimagnification convolutional neural networks for the determination and quantitation of lesions in nonclinical pathology studies.…”