Capturing Cellular Topology in Multi-Gigapixel Pathology Images

Lu, Wenqi; Graham, Simon; Bilal, Mohsin; Rajpoot, Nasir M.; Minhas, Fayyaz

doi:10.1109/cvprw50498.2020.00138

Cited by 60 publications

(41 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this task, we provide a top-performing approach for nuclear instance segmentation and classification within TIAToolbox, developed by members of our research group. The model, named HoVer-Net, has been increasingly used in recent research projects 4,11 in CPath, due to its state-of-the-art performance across a range of different datasets. In the toolbox, we include nuclear instance segmentation models trained on the PanNuke 41,42 , CoNSeP 1 and MoNuSAC 43 datasets – three widely used datasets for instance segmentation and classification of nuclei.…”

Section: Methodsmentioning

confidence: 99%

“…Digitization of classical cellular pathology workflows through the deployment of digital whole slide image (WSI) scanners has resulted in significant progress in the development of computational pathology (CPath) image analysis techniques. Such advances have benefited greatly by adapting deep learning techniques from computer vision producing novel solutions to a variety of CPath problems, including nucleus instance segmentation 1 , pathology image quality analysis 2 and WSI-level prediction 3,4 Although many algorithms have been developed for the analysis of WSIs which all share the same basic components (such as WSI reading, patch extraction and feeding to deep neural networks), there is no single open-source generic library that unifies all the steps using best practice to process these images. Several published algorithms have their own packaged codebases which run in a task-specific environment, with tightly coupled interfaces, dependencies and image format requirements.…”

Section: Mainmentioning

confidence: 99%

See 1 more Smart Citation

TIAToolbox: An End-to-End Toolbox for Advanced Tissue Image Analytics

Pocock

Graham

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Computational Pathology (CPath) has seen rapid growth in recent years, driven by advanced deep learning (DL) algorithms. These algorithms typically share the same sequence of steps. However, due to the sheer size and complexity of handling large multi-gigapixel whole-slide images, there is no open-source software library that provides a generic end-to-end API for pathology image analysis using best practices for CPath. Most researchers have designed custom pipelines from the bottom-up, restricting the development of advanced CPath algorithms to specialist users. To help overcome this bottleneck, we present TIAToolbox, a Python toolbox designed to make CPath more accessible to new and advanced CPath scientists and pathologists alike. We provide a usable and adaptable library with efficient, cutting-edge and unit-tested tools for data loading, pre-processing, model inference, post-processing and visualization. This enables all kinds of users to easily build upon recent DL developments in the CPath literature. TIAToolbox provides a user-friendly modular API to enable seamless integration of advanced DL algorithms. We show with the help of examples how state-of-the-art DL algorithms can be streamlined using TIAToolbox.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Mainmentioning

confidence: 99%

TIAToolbox: An End-to-End Toolbox for Advanced Tissue Image Analytics

Pocock

Graham

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In another example [11] a GCN is employed in order to perform prediction of the status of human epidermal growth factors; H&E stained slides of breast cancer were transformed into a graph; and the approach was demonstrated to be both more computationally efficient and more performant than prior state of the art.…”

Section: Related Workmentioning

confidence: 99%

An Unsupervised Graph Embeddings Approach to Multiplex Immunofluorescence Image Exploration

Innocenti

Zhang

Selvaraj

et al. 2021

Preprint

View full text Add to dashboard Cite

Understanding the complex biology of the tumor microenvironment (TME) is necessary to understand the mechanisms of action of immuno-oncology therapies and to match the right therapies to the right patients. Multiplex immunofluorescence (mIF) is a useful technology that has tremendous potential to further our understanding of cancer patho-biology; however, tools that fully leverage the high dimensionality of this data are still in their infancy. We describe here a novel deep learning pipeline aimed to allow Graph-based Inspection of Tissues via Embeddings, GraphITE. GraphITE transforms mIF data into a graph representation, where unsupervised learning algorithms can be utilised to generate embeddings representing cellular `neighbourhoods'. The embeddings can be downprojected and explored for clustering analysis, and patterns can be mapped back to the image as well as interrogated for phenotypical, morphological, or structural distinctiveness. GraphITE supports the extraction of information not only on the phenotypes of individual cells or the relationships between specific cell types, but is able to characterize cell neighborhoods to look for more complex interactions, thereby allowing pathologists and data scientists to explore mIF data sets, uncovering patterns that are otherwise obscured by the high-dimensionality of the data. In this work, we showcase the current setup of the system, going from raw input data all the way to a user friendly exploration tool. Using this tool, we show how the data can be navigated in a way previously not possible.

show abstract

“…Also, Adnan and colleagues developed a two-stage framework for WSI representation learning [14], where patches were sampled based on color and a graph neural network was constructed to learn the inter-patch relationships to discriminate lung adenocarcinoma (LUAD) from lung squamous cell carcinoma (LSCC). In another recent work, Lu and team developed a graph representation of the cellular architecture on the entire WSI to predict the status of human epidermal growth factor receptor 2 and progesterone receptor [15]. Their architecture attempted to create a bottom-up approach (i.e., nuclei- to WSI-level) to construct the graph, and in so doing, achieved a relatively efficient framework for analyzing the entire WSI.…”

Section: Introductionmentioning

confidence: 99%

A deep learning based graph-transformer for whole slide image classification

Zheng

Gindra

Betke

et al. 2021

Preprint

View full text Add to dashboard Cite

Deep learning is a powerful tool for assessing pathology data obtained from digitized biopsy slides. In the context of supervised learning, most methods typically divide a whole slide image (WSI) into patches, aggregate convolutional neural network outcomes on them and estimate overall disease grade. However, patch-based methods introduce label noise in training by assuming that each patch is independent with the same label as the WSI and neglect the important contextual information that is significant in disease grading. Here we present a Graph-Transformer (GT) based framework for processing pathology data, called GTP, that interprets morphological and spatial information at the WSI-level to predict disease grade. To demonstrate the applicability of our approach, we selected 3,024 hematoxylin and eosin WSIs of lung tumors and with normal histology from the Clinical Proteomic Tumor Analysis Consortium, the National Lung Screening Trial, and The Cancer Genome Atlas, and used GTP to distinguish adenocarcinoma (LUAD) and squamous cell carcinoma (LSCC) from those that have normal histology. Our model achieved consistently high performance on binary (tumor versus normal: mean overall accuracy = 0.975+/-0.013) as well as three-label (normal versus LUAD versus LSCC: mean accuracy = 0.932+/-0.019) classification on held-out test data, underscoring the power of GT-based deep learning for WSI-level classification. We also introduced a graph-based saliency mapping technique, called GraphCAM, that captures regional as well as contextual information and allows our model to highlight WSI regions that are highly associated with the class label. Taken together, our findings demonstrate GTP as a novel interpretable and effective deep learning framework for WSI-level classification.

show abstract

Capturing Cellular Topology in Multi-Gigapixel Pathology Images

Cited by 60 publications

References 36 publications

TIAToolbox: An End-to-End Toolbox for Advanced Tissue Image Analytics

TIAToolbox: An End-to-End Toolbox for Advanced Tissue Image Analytics

An Unsupervised Graph Embeddings Approach to Multiplex Immunofluorescence Image Exploration

A deep learning based graph-transformer for whole slide image classification

Contact Info

Product

Resources

About