Graph Neural Networks Ameliorate Potential Impacts of Imprecise Large-Scale Autonomous Immunofluorescence Labeling of Immune Cells on Whole Slide Images

Abstract

The characteristics of tumor-infiltrating lymphocytes (TIL) are essential in cancer prognostication and treatment through the ability to indicate the tumor's capacity to evade the immune system (e.g., as evidenced by nodal involvement). Machine learning technologies have demonstrated remarkable success for localizing TILs, though these methods require extensive curation of manual annotations or restaining procedures that can degrade tissue quality, resulting in imprecise annotation. In this study, we co-registered tissue slides stained for both hematoxylin and eosin (H\&E) and immunofluorescence (IF) as means to rapidly perform large-scale annotation of nuclei. We integrated the following approaches to improve the prediction of TILs: 1) minimized tissue degradation on same-section tissue restaining, 2) developed a scoring algorithm to improve the selection of patches for machine learning modeling and 3) utilized a graph neural network deep learning approach to identify relevant contextual features for lymphocyte prediction. Our graph neural network approach accounts for surrounding contextual micro/macro-architecture tissue features to facilitate interpretation of registered IF. The graph neural network compares favorably (F1-score=0.9235, AUROC=0.9462) to two alternative modeling approaches. This study brings insight to the importance of contextual information leveraged from within and around neighboring cells in a nuclei classification workflow, as well as elucidate approaches which enable the rapid generation of large-scale annotations of lymphocytes for machine learning approaches for immune phenotyping. Such approaches can help further interrogate the spatial biology of colorectal cancer tumors and tumor metastasis.

Integrative Co-Registration of Elemental Imaging and Histopathology for Enhanced Spatial Multimodal Analysis of Tissue Sections through TRACE

“…Alternatively, this process can be automated through the use of computer vision technologies, such as deep learning, to annotate tissue regions and cellular phenotypes (LeCun et al ., 2015). Deep learning algorithms excel at autonomously delineating both regions of interest within tissues and the constituent cell types, requiring minimal human oversight (Reddy et al ., 2022). Furthermore, H&E slides are considered the gold standard for aligning with multiplex immunofluorescence (mIF), multiplexed IHC (mIHC), and other spatial molecular modalities.…”

Feasibility of Inferring Spatial Transcriptomics from Single-Cell Histological Patterns for Studying Colon Cancer Tumor Heterogeneity

“…Cell graph neural networks (CGNN) facilitate the exchange of messages between adjacent cells, enabling the exchange/incorporation of contextual information [62][63][64][65][66][67]. This approach effectively captures the relationships between different cell populations within the tissue, including tumor cells and surrounding immune and other cell subpopulations.…”

Assessing the Impact of Pretraining Domain Relevance on Large Language Models Across Various Pathology Reporting Tasks

“…Deep learning (DL) methods, which are computational heuristics inspired by processes of the central nervous system, excel at processing imaging data to predict the risk of lung cancer 5 , recognize melanoma within dermoscopic images 6 , and segment digitized kidney tissue sections 7 , automatically detect early signs of colorectal cancer during colonoscopies 8 , and more recently to flexibly encode and interpret biomedical data including clinical language, imaging, and genomics 9 , amongst other tasks. Through the use of specialized and updatable image filters/shapes, which are used to localize imaging features through their optimal alignment, these algorithms are commonly used for binary and multi-class classification tasks 1011 , and the localization of heterogenous cell lineages within distinct spatial architectures to inform the pathological assessment 12 .…”