Deep Instance-Level Hard Negative Mining Model for Histopathology Images

Abstract

Histopathology image analysis can be considered as a Multiple instance learning (MIL) problem, where the whole slide histopathology image (WSI) is regarded as a bag of instances (i.e., patches) and the task is to predict a single class label to the WSI. However, in many reallife applications such as computational pathology, discovering the key instances that trigger the bag label is of great interest because it provides reasons for the decision made by the system. In this paper, we propose a deep convolutional neural network (CNN) model that addresses the primary task of a bag classification on a histopathology image and also learns to identify the response of each instance to provide interpretable results to the final prediction. We incorporate the attention mechanism into the proposed model to operate the transformation of instances and learn attention weights to allow us to find key patches. To perform a balanced training, we introduce adaptive weighing in each training bag to explicitly adjust the weight distribution in order to concentrate more on the contribution of hard samples. Based on the learned attention weights, we further develop a solution to boost the classification performance by generating the bags with hard negative instances. We conduct extensive experiments on colon and breast cancer histopathology data and show that our framework achieves state-of-the-art performance.

Deep Learning for Necrosis Detection Using Canine Perivascular Wall Tumour Whole Slide Images

“…Hard negative mining was performed in order to reduce the number of false positives. This is an approach that allowed us to train the model with additional "difficult" examples presented to the network 24 . Firstly, a "full" training set was created, where we extracted every single negative class patch from the training set.…”

A Framework for Falsifiable Explanations of Machine Learning Models with an Application in Computational Pathology

“…This provides a practical and crucial advantage over most post-hoc methods, which require backpropagation steps that are more difficult to Some existing machine learning models implicitly match parts of our definition of falsifiable explanations. For example, applying multiple instance learning in histopathology [Campanella et al, 2019, Lu et al, 2021, Ilse et al, 2018, Li et al, 2019 tiles whole-slide samples into a large number of patches, and the key task is to identify the few disease-relevant patches among this large number. The few positive instances claim to localize disease, which yields a basis for a falsifiable hypothesis.…”

Fibrosis severity scoring on Sirius red histology with multiple-instance deep learning

“…MIL frameworks rely on specific strategies for tile selection, feature extraction, and multiple inference aggregation. Common MIL paradigms rely on one training "bag" per image after careful tile selection (e.g., tissue content above a threshold) (12)(13)(14) ; data augmentations (12)(13)(14)(15)(16)(17) ; use of transfer learning from ImageNet via GoogLeNet, InceptionNet, ResNet, and MobileNet (18,19) for cancer, and AlexNet and ResNet for NAFLD (20,21) ; aggregation of tile-level inferences via max-pooling, or of tile-level features via average-pooling, attention-based or RNN based frameworks ahead of WSI-level prediction (12,13,16,21,22) .…”

“…Bags are composed of a pre-fixed number of k tiles and are given the original WSI label as a proxy groundtruth label. The majority of existing literature constructs a single bag per WSI that includes from thousands to tens of thousands of tiles with deemed-sufficient tissue content (12)(13)(14)(15)(16) . Unlike most cancer focused datasets, the pathological signs in NAFLD are sparse.…”