Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images

Abstract

Nuclear segmentation and classification within Haematoxylin & Eosin stained histology images is a fundamental prerequisite in the digital pathology work-flow. The development of automated methods for nuclear segmentation and classification enables the quantitative analysis of tens of thousands of nuclei within a whole-slide pathology image, opening up possibilities of further analysis of large-scale nuclear morphometry. However, automated nuclear segmentation and classification is faced with a major challenge in that there are several different types of nuclei, some of them exhibiting large intra-class variability such as the nuclei of tumour cells. Additionally, some of the nuclei are often clustered together. To address these challenges, we present a novel convolutional neural network for simultaneous nuclear segmentation and classification that leverages the instance-rich information encoded within the vertical and horizontal distances of nuclear pixels to their centres of mass. These distances are then utilised to separate clustered nuclei, resulting in an accurate segmentation, particularly in areas with overlapping instances. Then, for each segmented instance the network predicts the type of nucleus via a devoted up-sampling branch. We demonstrate state-of-the-art performance compared to other methods on multiple independent multi-tissue histology image datasets. As part of this work, we introduce a new dataset of Haematoxylin & Eosin stained colorectal adenocarcinoma image tiles, containing 24,319 exhaustively annotated nuclei with associated class labels.

Enhancing Multi-tissue and Multi-scale Cell Nuclei Segmentation with Deep Metric Learning

“…However, most of these algorithms are applicable only for the separation of nuclei, which have an ellipse-like shape [40]. In general, we infer that the mean value 0.764 of Dice similarity coefficient achieved in the current paper for the large-scale multi-tissue collection of images is in line with already published studies, where its value varied from 0.69 to 0.9 when the deep learning model was applied [6,41,42]. However, the comparison of the obtained results is not trivial because of the different algorithms applied and the accuracy metrics selected, as well as the different image qualities, magnifications (40× or 20×) and different staining techniques, etc., in the dataset.…”

Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images

“…In the bottom part of Table II, we compare our proposed approach with MILD-Net [14] and Rota-Net [29], which are top-performing gland segmentation methods and therefore can be appropriately used for performance benchmarking. Like [43] {e} 29.4M 0.811 0.407 U-Net [38] {e} 37.0M 0.758 0.478 Mask-RCNN [44] {e} 40.1M 0.760 0.509 DIST [17] {e} 9.2M 0.789 0.443 Micro-Net [45] {e} 192.6M 0.797 0.519 CIA-Net [46] {e} 22.0M 0.818 0.577 HoVer-Net [16] {e} 54.7M 0.826 0.597 DSF-CNN (Ours) C 8 3.7M 0.826 0.600 the p4m-DenseNet, Rota-Net makes use of the standard Gconvolution, but is limited to only 90 • filter rotations. In addition, we compare with FCN8 and U-Net as they are two widely used CNNs for segmentation.…”

“…We evaluate the performance of our proposed method with several state-of-the-art approaches in the bottom part of Table III. In particular, HoVer-Net [16], CIA-Net [46], Micro-Net [45] and DIST [17] have been purpose-built for the task of nuclear segmentation and, therefore, provide a competitive benchmark. The proposed DSF-CNN once again achieves the best performance compared to other methods for both binary DICE and panoptic quality, on par with the state-of-the-art HoVer-Net method, while requiring a fraction of the parameter count.…”

“…Since the increase in computing power and the development of algorithms that assist network optimisation [10] CNNs have become deeper [7], [11], leading to current state-of-theart performance in numerous image recognition tasks [12], [13]. As a result of the success of deep learning, CNNs have since been widely used in CPath for various tasks including: gland segmentation [14], [15]; nucleus segmentation [16]- [18]; mitosis detection [19]; cancer type prediction [20] and cancer grading [21], [22]. Yet, unlike translation, CNNs do not behave well with respect to rotation because this symmetry is not built into the network architecture.…”

CGC-Net: Cell Graph Convolutional Network for Grading of Colorectal Cancer Histology Images

“…Colorectal nuclear segmentation and phenotypes (CoN-SeP) dataset: To train the nuclear segmentation network for graph construction, we use the CoNSeP dataset [23]. CoNSeP consists of 41 H&E stained images with 1000×1000 pixels at 40× magnification extracted from 16 CRA WSIs.…”