Mixed Effects Machine Learning Models for Colon Cancer Metastasis Prediction using Spatially Localized Immuno-Oncology Markers

Levy, Joshua; Bobak, Carly A.; Nasir-Moin, Mustafa; Veziroglu, Eren M.; Palisoul, Scott; Barney, Rachael E.; Salas, Lucas A.; Christensen, Brock C.; Tsongalis, Gregory J.; Vaickus, Louis

doi:10.1142/9789811250477_0017

Cited by 7 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The primary dataset utilized in this study was acquired from four pathologic T Stage-III (pT3) matched (pTNM system) colorectal cancer patients at Dartmouth Hitchcock Medical Center, determined through a retrospective review of pathology reports from 2016 to 2019 following IRB review and approval. These four patients were drawn from a set of 36 patients included in a prior study 15 – half of the patients had concurrent tumor metastasis (slides A1 and B1 had tumor metastasis; slides C1 and D1 did not have tumor metastasis) and were otherwise matched on age, sex, tumor grade, tissue size, mismatch repair (MMR) status, and tumor site using iterative patient resampling with t-tests for continuous variables and fisher’s exact tests for categorical variables. The four patients were subselected to restrict the tumor site (three in the right colon, one in the transverse colon), grade (three grade 1, one grade 2), node status (two metastasis cases with N-1a), and account for differences in sex (50% female within both the non-metastasis and metastasis groups).…”

Section: Methodsmentioning

confidence: 99%

Inferring Spatially Resolved Transcriptomics Data from Whole Slide Images for the Assessment of Colorectal Tumor Metastasis: A Feasibility Study

Fatemi

Feng²,

Sharma

et al. 2022

Preprint

View full text Add to dashboard Cite

Over 150,000 Americans are diagnosed with colorectal cancer (CRC) every year, and annually over 50,000 individuals will die from CRC, necessitating improvements in screening, prognostication, disease management, and therapeutic options. Tumor metastasis is the primary factor related to the risk of recurrence and mortality. Yet, screening for nodal and distant metastasis is costly, and invasive and incomplete resection may hamper adequate assessment. Signatures of the tumor-immune microenvironment (TIME) at the primary site can provide valuable insights into the aggressiveness of the tumor and the effectiveness of various treatment options. Spatially-resolved transcriptomics technologies offer an unprecedented characterization of TIME through high multiplexing, yet their scope is constrained by cost. Meanwhile, it has long been suspected that histological, cytological and macroarchitectural tissue characteristics correlate well with molecular information (e.g., gene expression). Thus, a method for predicting transcriptomics data through inference of RNA patterns from whole slide images (WSI) is a key step in studying metastasis at scale. In this work, we collected and preprocessed Visium spatial transcriptomics data (17,943 genes at up to 5,000 spots per patient sampled in a honeycomb pattern) from tissue across four stage-III matched colorectal cancer patients. We compare and prototype several convolutional, Transformer, and graph convolutional neural networks to predict spatial RNA patterns under the hypothesis that the transformer and graph-based approaches better capture relevant spatial tissue architecture. We further analyzed the model’s ability to recapitulate spatial autocorrelation statistics using SPARK and SpatialDE. Overall, results indicate that the transformer and graph-based approaches were unable to outperform the convolutional neural network architecture, though they exhibited optimal performance for relevant disease-associated genes. Initial findings suggest that different neural networks that operate on different scales are relevant for capturing distinct disease pathways (e.g., epithelial to mesenchymal transition). We add further evidence that deep learning models can accurately predict gene expression in whole slide images and comment on understudied factors which may increase its external applicability (e.g., tissue context). Our preliminary work will motivate further investigation of inference for molecular patterns from whole slide images as metastasis predictors and in other applications.

show abstract

Section: Methodsmentioning

confidence: 99%

Inferring Spatially Resolved Transcriptomics Data from Whole Slide Images for the Assessment of Colorectal Tumor Metastasis: A Feasibility Study

Fatemi

Feng²,

Sharma

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…level scores and other demographic/specimen characteristics into an Atypia Burden Score (ABS), accounting for repeat measures by patient [58][59][60][61][62][63][64] .…”

Section: Classifier Development-machine Learning Classifier Which Int...mentioning

confidence: 99%

Large-Scale Validation Study of an Improved Semi-Autonomous Urine Cytology Assessment Tool: AutoParis-X

Levy

Chan

Marotti

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Adopting a computational approach for the assessment of urine cytology specimens has the potential to improve the efficiency, accuracy and reliability of bladder cancer screening, which has heretofore relied on semi-subjective manual assessment methods. As rigorous, quantitative criteria and guidelines have been introduced for improving screening practices, e.g., The Paris System for Reporting Urinary Cytology (TPS), algorithms to emulate semi-autonomous diagnostic decision-making have lagged behind, in part due to the complex and nuanced nature of urine cytology reporting. In this study, we report on a deep learning tool, AutoParis-X, which can facilitate rapid semi-autonomous examination of urine cytology specimens. Through a large-scale retrospective validation study, results indicate that AutoParis-X can accurately determine urothelial cell atypia and aggregate a wide-variety of cell and cluster-related information across a slide to yield an Atypia Burden Score (ABS) that correlates closely with overall specimen atypia, predictive of TPS diagnostic categories. Importantly, this approach accounts for challenges associated with assessment of overlapping cell cluster borders, which improved the ability to predict specimen atypia and accurately estimate the nuclear-to-cytoplasm (NC) ratio for cells in these clusters. We developed an interactive web application that is publicly available and open-source, which features a simple, easy-to-use display for examining urine cytology whole-slide images (WSI) and determining the atypia level of specific cells, flagging the most abnormal cells for pathologist review. The accuracy of AutoParis-X (and other semi-automated digital pathology systems) indicates that these technologies are approaching clinical readiness and necessitates full evaluation of these algorithms via head-to-head clinical trials.

show abstract

“…Sections were stained with fluorescent-labeled, IF, antibodies for the following markers: 1) tumor/epithelial (PanCK), immune cells (CD45), and nuclei (SYTO13). These IF stains were initially acquired for a previously published study on spatial immune markers of metastasis, which utilized the GeoMX Digital Spatial Profiler (DSP, Nanostring Technologies, Seattle, WA) for image scanning into 16-bit unsigned color (one channel per stain) TIFF format images (Levy et al, 2022). After IF staining, the same sections were stained for H&E (without requiring destaining as the chemical reagents of the H&E minimally interacted with the fluorophores) and scanned into WSI using the Aperio AT2 scanner at 20x (8-bit unsigned color).…”

Section: Data Collectionmentioning

confidence: 99%

Graph Neural Networks Ameliorate Potential Impacts of Imprecise Large-Scale Autonomous Immunofluorescence Labeling of Immune Cells on Whole Slide Images

Reddy

Sharma

et al. 2022

Preprint

View full text Add to dashboard Cite

The characteristics of tumor-infiltrating lymphocytes (TIL) are essential in cancer prognostication and treatment through the ability to indicate the tumor's capacity to evade the immune system (e.g., as evidenced by nodal involvement). Machine learning technologies have demonstrated remarkable success for localizing TILs, though these methods require extensive curation of manual annotations or restaining procedures that can degrade tissue quality, resulting in imprecise annotation. In this study, we co-registered tissue slides stained for both hematoxylin and eosin (H\&E) and immunofluorescence (IF) as means to rapidly perform large-scale annotation of nuclei. We integrated the following approaches to improve the prediction of TILs: 1) minimized tissue degradation on same-section tissue restaining, 2) developed a scoring algorithm to improve the selection of patches for machine learning modeling and 3) utilized a graph neural network deep learning approach to identify relevant contextual features for lymphocyte prediction. Our graph neural network approach accounts for surrounding contextual micro/macro-architecture tissue features to facilitate interpretation of registered IF. The graph neural network compares favorably (F1-score=0.9235, AUROC=0.9462) to two alternative modeling approaches. This study brings insight to the importance of contextual information leveraged from within and around neighboring cells in a nuclei classification workflow, as well as elucidate approaches which enable the rapid generation of large-scale annotations of lymphocytes for machine learning approaches for immune phenotyping. Such approaches can help further interrogate the spatial biology of colorectal cancer tumors and tumor metastasis.

show abstract

Mixed Effects Machine Learning Models for Colon Cancer Metastasis Prediction using Spatially Localized Immuno-Oncology Markers

Cited by 7 publications

References 0 publications

Inferring Spatially Resolved Transcriptomics Data from Whole Slide Images for the Assessment of Colorectal Tumor Metastasis: A Feasibility Study

Inferring Spatially Resolved Transcriptomics Data from Whole Slide Images for the Assessment of Colorectal Tumor Metastasis: A Feasibility Study

Large-Scale Validation Study of an Improved Semi-Autonomous Urine Cytology Assessment Tool: AutoParis-X

Graph Neural Networks Ameliorate Potential Impacts of Imprecise Large-Scale Autonomous Immunofluorescence Labeling of Immune Cells on Whole Slide Images

Contact Info

Product

Resources

About