Deep learning for diagnosis and survival prediction in soft tissue sarcoma

Abstract

Background: Clinical management of soft tissue sarcoma (STS) is particularly challenging. Here, we used digital pathology and deep learning (DL) for diagnosis and prognosis prediction of STS. Patients and methods: Our retrospective, multicenter study included a total of 506 histopathological slides from 291 patients with STS. The Cancer Genome Atlas cohort (240 patients) served as training and validation set. A second, multicenter cohort (51 patients) served as an additional test set. The use of the DL model (DLM) as a clinical decision support system was evaluated by nine pathologists with different levels of expertise. For prognosis prediction, 139 slides from 85 patients with leiomyosarcoma (LMS) were used. Area under the receiver operating characteristic (AUROC) and accuracy served as main outcome measures. Results: The DLM achieved a mean AUROC of 0.97 (AE0.01) and an accuracy of 79.9% (AE6.1%) in diagnosing the five most common STS subtypes. The DLM significantly improved the accuracy of the pathologists from 46.3% (AE15.5%) to 87.1% (AE11.1%). Furthermore, they were significantly faster and more certain in their diagnosis. In LMS, the mean AUROC in predicting the disease-specific survival status was 0.91 (AE0.1) and the accuracy was 88.9% (AE9.9%). Cox regression showed the DLM's prediction to be a significant independent prognostic factor (P ¼ 0.008, hazard ratio 5.5, 95% confidence interval 1.56-19.7) in these patients, outperforming other risk factors. Conclusions: DL can be used to accurately diagnose frequent subtypes of STS from conventional histopathological slides. It might be used for prognosis prediction in LMS, the most prevalent STS subtype in our cohort. It can also help pathologists to make faster and more accurate diagnoses. This could substantially improve the clinical management of STS patients.

Multimodal Deep Learning for Prognosis Prediction in Renal Cancer

“…For pathological tasks for example, we were able to predict the molecular subtype of muscle-invasive bladder cancer from conventional histopathological slides alone using deep learning (DL) ( 9 ). We also used a similar approach for prognosis prediction in soft tissue sarcoma (STS) ( 10 ). But while it is technically feasible, there are very few studies so far evaluating the use of multimodal input for training of AI and DL models ( 11 ).…”

Artificial intelligence to identify genetic alterations in conventional histopathology

“…This includes radiology, dermatoscopy, and histopathology image analysis. In prognostication, DL has been shown to outperform some established risk factors in colorectal cancer [26][27][28], breast cancer [27], lung cancer [29], sarcoma [30], bladder cancer [31], glioma [32], mesothelioma [33], and hepatocellular carcinoma [34,35], among other tumor types. Compared with prognostic patterns in conventional histology slides, morphological patterns reflecting specific molecular alterations are generally weaker.…”

DeepMed: A unified, modular pipeline for end-to-end deep learning in computational pathology

“…For example, many methods have been designed to train CNNs on a training set and test them on a designated test set. 9 Others have used stratified cross-validation 4,13 or Monte-Carlo cross-validation 14 on a patient-level. Moving from one experimental design to another requires a multitude of upstream and downstream changes, related to data preprocessing, statistical metrics, visualization and essentially any component of the pipeline.…”

“…DeepMed includes all commonly used variants of data loading, network training, statistics and visualization in a fully modular way (Figure 1C). DeepMed can be used for a wide range of problems including: simple classification and regression tasks on histological image data only, 3,9,13,[15][16][17][18] , prediction of survival markers 19 (Figure 1D) and inclusion of additional non-image in the training process in a multi-modal way 20 (Figure 1E). DeepMed enables researchers to conveniently use established methods and test dozens of hypotheses in a single cohort or multiple patient cohorts with minimal data preprocessing.…”