Predicting the HER2 status in oesophageal cancer from tissue microarrays using convolutional neural networks

Abstract

Background Fast and accurate diagnostics are key for personalised medicine. Particularly in cancer, precise diagnosis is a prerequisite for targeted therapies, which can prolong lives. In this work, we focus on the automatic identification of gastroesophageal adenocarcinoma (GEA) patients that qualify for a personalised therapy targeting epidermal growth factor receptor 2 (HER2). We present a deep-learning method for scoring microscopy images of GEA for the presence of HER2 overexpression. Methods Our method is based on convolutional neural networks (CNNs) trained on a rich dataset of 1602 patient samples and tested on an independent set of 307 patient samples. We additionally verified the CNN’s generalisation capabilities with an independent dataset with 653 samples from a separate clinical centre. We incorporated an attention mechanism in the network architecture to identify the tissue regions, which are important for the prediction outcome. Our solution allows for direct automated detection of HER2 in immunohistochemistry-stained tissue slides without the need for manual assessment and additional costly in situ hybridisation (ISH) tests. Results We show accuracy of 0.94, precision of 0.97, and recall of 0.95. Importantly, our approach offers accurate predictions in cases that pathologists cannot resolve and that require additional ISH testing. We confirmed our findings in an independent dataset collected in a different clinical centre. The attention-based CNN exploits morphological information in microscopy images and is superior to a predictive model based on the staining intensity only. Conclusions We demonstrate that our approach not only automates an important diagnostic process for GEA patients but also paves the way for the discovery of new morphological features that were previously unknown for GEA pathology.

Deep Neural Networks for HER2 Grading of Whole Slide Images with Subclasses Levels

“…The remaining networks, ResNet and DenseNet were selected based on their ability to offer new attributes, achieving good results while maintaining a trade-off between computational time and workload. In addition, ResNet has been frequently used for HER2 scoring in breast cancer [18,20,22] and more recently for predicting the HER2 status in oesophageal cancer [32]. Moreover, DenseNet has also been employed recently for HER2 scoring in breast cancer [5].…”

Teacher-student collaborated multiple instance learning for pan-cancer PDL1 expression prediction from histopathology slides

“…However, the dataset size could be constrained due to the labor-intensive nature of annotating such data. Some other approaches have opted to overlook the intra-tumor heterogeneity by assigning slide-level annotations to all content within a slide 31 , 34 , 35 , 41 , 42 . This strategy works well if the slides exhibit good homogeneity concerning specific properties of interest.…”

PhiHER2: phenotype-informed weakly supervised model for HER2 status prediction from pathological images

“…Both of them achieved HER2 status prediction, but they are equipped with a full supervised learning strategy, causing a tremendous computational cost on billions of patches, compared to weakly supervised learning methods. Pisula et al (2023) implemented a weakly supervised multiple instance learning (MIL) method for IHC-stained tissue slides and an attention-based MIL approach ReceptorNet ( Naik et al 2020 ) was also performed for hormonal receptor and HER2 status prediction on H&E WSIs. Even, a graph neural network architecture was introduced for HER2 status prediction from WSIs of BC tissue ( Lu et al 2022 ).…”