A graph neural network framework for mapping histological topology in oral mucosal tissue

Abstract

Background Histological feature representation is advantageous for computer aided diagnosis (CAD) and disease classification when using predictive techniques based on machine learning. Explicit feature representations in computer tissue models can assist explainability of machine learning predictions. Different approaches to feature representation within digital tissue images have been proposed. Cell-graphs have been demonstrated to provide precise and general constructs that can model both low- and high-level features. The basement membrane is high-level tissue architecture, and interactions across the basement membrane are involved in multiple disease processes. Thus, the basement membrane is an important histological feature to study from a cell-graph and machine learning perspective. Results We present a two stage machine learning pipeline for generating a cell-graph from a digital H &E stained tissue image. Using a combination of convolutional neural networks for visual analysis and graph neural networks exploiting node and edge labels for topological analysis, the pipeline is shown to predict both low- and high-level histological features in oral mucosal tissue with good accuracy. Conclusions Convolutional and graph neural networks are complementary technologies for learning, representing and predicting local and global histological features employing node and edge labels. Their combination is potentially widely applicable in histopathology image analysis and can enhance explainability in CAD tools for disease prediction.

Combining graph neural networks and computer vision methods for cell nuclei classification in lung tissue

“…Alternatively, graph representations of histology slides have been proposed to overcome the aforementioned CNN failure cases [10] , [11] , [12] , [13] , [14] , [15] , [16] , [17] , [18] , [19] . Specifically, Cell Graphs (CGs) can be constructed by detecting with a CNN the cell nuclei in images.…”

“…Specifically, Cell Graphs (CGs) can be constructed by detecting with a CNN the cell nuclei in images. These cell nuclei form the nodes of the graph and can be associated with categorical information [16] , [17] . The edges of the graph can be created based on nearest neighbours [12] , [14] or using the Delaunay triangulation to obtain a planar graph [16] , [17] .…”

“…These cell nuclei form the nodes of the graph and can be associated with categorical information [16] , [17] . The edges of the graph can be created based on nearest neighbours [12] , [14] or using the Delaunay triangulation to obtain a planar graph [16] , [17] . Moreover, because of their recent success for graph-structured data analysis, Graph Neural Networks (GNNs) have been applied to process these CGs.…”

“…Moreover, because of their recent success for graph-structured data analysis, Graph Neural Networks (GNNs) have been applied to process these CGs. Some works choose the architecture of GraphSAGE [12] , [16] while others employ the Graph Isomorphism Network (GIN) [14] . We test both on our dataset and expand the comparison to graph convolution and attention [20] , [21] .…”

Spatial Morphoproteomic Features Predict Uniqueness of Immune Microarchitectures and Responses in Lymphoid Follicles

“…Graph neural networks (GNN) have shown ground-breaking performance on many deep learning tasks 8 . Modeling tumor microenvironment single-cell spatial organization as graphs and applying GNN have predicted response and survival at the patient level using various types of imaging modalities such as multiplex imaging 9,10 and H&E 11,12 . On the other hand, cell morphology analysis has been used to study the classification of cell phenotypes 13,14 .…”