WSISA: Making Survival Prediction from Whole Slide Histopathological Images

Zhu, Xiaofan; Yao, Jiawen; Zhu, Fayan; Huang, Junzhou

doi:10.1109/cvpr.2017.725

Cited by 235 publications

(178 citation statements)

References 26 publications

Supporting

Mentioning

160

Contrasting

Order By: Relevance

“…Nowadays, survival analysis has been widely used in realworld applications, such as clinical analysis in medicine research (Zhu et al 2017b;Luck et al 2017;Katzman et al 2018) taking diseases as events and predicting survival time of patients; customer lifetime estimation in information systems (Jing and Smola 2017;Grob et al 2018) which estimates the time until the next visit of users; market modeling in game theory fields (Wu, Yeh, and Chen 2015;Wang et al 2016) that predicts the event (i.e., winning) probability over the whole referral space.…”

Section: Introductionmentioning

confidence: 99%

Deep Recurrent Survival Analysis

Ren

Qin

Zheng

et al. 2019

AAAI

View full text Add to dashboard Cite

Survival analysis is a hotspot in statistical research for modeling time-to-event information with data censorship handling, which has been widely used in many applications such as clinical research, information system and other fields with survivorship bias. Many works have been proposed for survival analysis ranging from traditional statistic methods to machine learning models. However, the existing methodologies either utilize counting-based statistics on the segmented data, or have a pre-assumption on the event probability distribution w.r.t. time. Moreover, few works consider sequential patterns within the feature space. In this paper, we propose a Deep Recurrent Survival Analysis model which combines deep learning for conditional probability prediction at finegrained level of the data, and survival analysis for tackling the censorship. By capturing the time dependency through modeling the conditional probability of the event for each sample, our method predicts the likelihood of the true event occurrence and estimates the survival rate over time, i.e., the probability of the non-occurrence of the event, for the censored data. Meanwhile, without assuming any specific form of the event probability distribution, our model shows great advantages over the previous works on fitting various sophisticated data distributions. In the experiments on the three realworld tasks from different fields, our model significantly outperforms the state-of-the-art solutions under various metrics.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Recurrent Survival Analysis

Ren

Qin

Zheng

et al. 2019

AAAI

View full text Add to dashboard Cite

show abstract

“…9 It is a good way to learn highly complex survival functions by using the advanced neural networks techniques. 10,12 We can get the risk score through neural networks and now denote the risk for the patient i as o i . Deepsurv 10 is the earlier attempt to learn a nonlinear risk function by replacing the linear part β T x in f (x) with a nonlinear deep fully connected network.…”

Section: Survival Analysismentioning

confidence: 99%

“…[10][11][12] Katzman et al proposed a deep fully-connected network (DeepSurv) to represent the nonlinear risk function. 10 They demonstrated that DeepSurv outperformed the standard linear Cox proportional hazard model.…”

Section: Introductionmentioning

confidence: 99%

Deep Integrative Analysis for Survival Prediction

2017

View full text Add to dashboard Cite

Survival prediction is very important in medical treatment. However, recent leading research is challenged by two factors: 1) the datasets usually come with multi-modality; and 2) sample sizes are relatively small. To solve the above challenges, we developed a deep survival learning model to predict patients' survival outcomes by integrating multi-view data. The proposed network contains two sub-networks, one view-specific and one common sub-network. We designated one CNN-based and one FCN-based sub-network to efficiently handle pathological images and molecular profiles, respectively. Our model first explicitly maximizes the correlation among the views and then transfers feature hierarchies from view commonality and specifically fine-tunes on the survival prediction task. We evaluate our method on real lung and brain tumor data sets to demonstrate the effectiveness of the proposed model using data with multiple modalities across different tumor types.

show abstract

“…microRNA or mRNA) 26 and high resolution histopathology whole slide images (WSIs). Yet, based on previous 27 work, only a subset of the genomic image features are relevant for predicting prognosis. 28 Thus, to successfully develop a multi-modal model for prognosis prediction, an approach 29 is required that can efficiently work with clinical, genomic and image data, in essence 30 multimodal data.…”

mentioning

confidence: 99%

“…However, in prognosis prediction, 78 truly-automated WSI-based systems have had limited success. One report uses a 79 slide-based approach that relies on unsupervised learning -Zhu et al's recent paper 80 uses K-means clustering to characterize and adaptively sample patches within slide 81 images, achieving 0.708 C-index on lung cancer data [27], a result that nearly rivals 82 genomic-data approaches.…”

mentioning

confidence: 99%

Deep Learning with Multimodal Representation for Pancancer Prognosis Prediction

Cheerla¹,

Gevaert

2019

Preprint

View full text Add to dashboard Cite

Estimating the future course of cancer is invaluable to physicians; however, current clinical methods fail to effectively use the vast amount of multimodal data that is available for cancer patients.To tackle this problem, we constructed a deep neural network based model to predict the survival of patients for 20 different cancer types using gene expressions, microRNA data, clinical data and histopathology whole slide images (WSIs). We developed an unsupervised encoder to compress these four data modalities into a single feature vector for each patient, handling missing data through a resilient, multimodal dropout method. Encoding methods were tailored to each data type -using deep highway networks to extract features from genomic and clinical data, and convolutional neural networks extract features from pathology images. We then used these feature encodings trained on pancancer data to predict pancancer and single cancer survival data, achieving a C-index of 0.784 overall.This work shows that it is possible to build a pancancer model for prognosis that also predicts prognosis in single cancer sites. Furthermore, our model handles multiple data modalities, efficiently analyzes WSIs, and summarizes patient details flexibly into an unsupervised, informative profile. We thus present a powerful automated tool to accurately determine prognosis, a key step towards personalized treatment for cancer patients.

show abstract

WSISA: Making Survival Prediction from Whole Slide Histopathological Images

Cited by 235 publications

References 26 publications

Deep Recurrent Survival Analysis

Deep Recurrent Survival Analysis

Deep Integrative Analysis for Survival Prediction

Deep Learning with Multimodal Representation for Pancancer Prognosis Prediction

Contact Info

Product

Resources

About