Summary The determination of endometrial carcinoma histological subtypes, molecular subtypes, and mutation status is critical for the diagnostic process, and directly affects patients’ prognosis and treatment. Sequencing, albeit slower and more expensive, can provide additional information on molecular subtypes and mutations that can be used to better select treatments. Here, we implement a customized multi-resolution deep convolutional neural network, Panoptes, that predicts not only the histological subtypes but also the molecular subtypes and 18 common gene mutations based on digitized H&E-stained pathological images. The model achieves high accuracy and generalizes well on independent datasets. Our results suggest that Panoptes, with further refinement, has the potential for clinical application to help pathologists determine molecular subtypes and mutations of endometrial carcinoma without sequencing.

Background The COVID-19 pandemic began in early 2021 and placed significant strains on health care systems worldwide. There remains a compelling need to analyze factors that are predictive for patients at elevated risk of morbidity and mortality. Objective The goal of this retrospective study of patients who tested positive with COVID-19 and were treated at NYU (New York University) Langone Health was to identify clinical markers predictive of disease severity in order to assist in clinical decision triage and to provide additional biological insights into disease progression. Methods The clinical activity of 3740 patients at NYU Langone Hospital was obtained between January and August 2020; patient data were deidentified. Models were trained on clinical data during different parts of their hospital stay to predict three clinical outcomes: deceased, ventilated, or admitted to the intensive care unit (ICU). Results The XGBoost (eXtreme Gradient Boosting) model that was trained on clinical data from the final 24 hours excelled at predicting mortality (area under the curve [AUC]=0.92; specificity=86%; and sensitivity=85%). Respiration rate was the most important feature, followed by SpO2 (peripheral oxygen saturation) and being aged 75 years and over. Performance of this model to predict the deceased outcome extended 5 days prior, with AUC=0.81, specificity=70%, and sensitivity=75%. When only using clinical data from the first 24 hours, AUCs of 0.79, 0.80, and 0.77 were obtained for deceased, ventilated, or ICU-admitted outcomes, respectively. Although respiration rate and SpO2 levels offered the highest feature importance, other canonical markers, including diabetic history, age, and temperature, offered minimal gain. When lab values were incorporated, prediction of mortality benefited the most from blood urea nitrogen and lactate dehydrogenase (LDH). Features that were predictive of morbidity included LDH, calcium, glucose, and C-reactive protein. Conclusions Together, this work summarizes efforts to systematically examine the importance of a wide range of features across different endpoint outcomes and at different hospitalization time points.

Determining endometrial carcinoma histological subtype is a critical diagnostic process that directly affects prognosis and treatment options. Recently, molecular subtyping and mutation status are gaining popularity as they offer more relevant information to evaluate the severity and develop individualized therapies. However, compared to the histopathological approach, the availability of molecular subtyping is limited as it can only be obtained by genomic sequencing, which is more expensive. Here, we implemented deep convolutional neural network models that predict not only the histological subtypes, but also molecular subtypes and 18 common gene mutations based on digitized H&E stained pathological images. Taking advantage of the multiresolution nature of the whole slide images, we introduced a customized architecture, Panoptes, to integrate features of different magnification. The model was trained and evaluated with images from The Cancer Genome Atlas and Clinical Proteomic Tumor Analysis Consortium. Our models achieved an area under the receiver operating characteristic curve (AUROC) of 0.969 in predicting histological subtype and 0.934 to 0.958 in predicting CNV-H molecular subtype. The prediction tasks of 4 mutations and MSI-High molecular subtype also achieved high performance with AUROC ranging from 0.781 to 0.873. Panoptes showed significantly better performance than InceptionResnet in most of these top predicted tasks by up to 18%. Feature extraction and visualization revealed that the model relied on human-interpretable patterns. Our results suggest that Panoptes can help pathologists determine molecular subtypes and mutations without sequencing, and is generally applicable to any cancer type.

Studies have shown that STK11 mutation plays a critical role in affecting the lung adenocarcinoma (LUAD) tumor immune environment. By training an Inception-Resnet-v2 deep convolutional neural network model, we were able to classify STK11-mutated and wild type LUAD tumor histopathology images with a promising accuracy (per slide AUROC=0.795). Dimensional reduction of the activation maps before the output layer of the test set images revealed that less immune cells were accumulated around cancer cells in STK11-mutation cases. Our study demonstrated that deep convolutional network model can automatically identify STK11 mutations based on histopathology slides and confirmed that the immune cell density was the main feature used by the model to distinguish STK11-mutated cases.