“…Notable examples include methods designed to use clinical and gene expression data, namely DeepSurv 13 and Cox-nnet 14 . Other methods focused on imaging data, such as CXR-risk, which uses chest radiographs 15 , LungNet 16 and a gastric cancer survival prediction model 17 , which use computed tomography (CT) images, a nasopharyngeal carcinoma survival prediction model 18 , which uses magnetic resonance imaging (MRI) data, and WSISA 19 , which employs histopathology slides.…”
The age of precision medicine demands powerful computational techniques to handle high-dimensional patient data. We present MultiSurv, a multimodal deep learning method for long-term pan-cancer survival prediction. MultiSurv uses dedicated submodels to establish feature representations of clinical, imaging, and different high-dimensional omics data modalities. A data fusion layer aggregates the multimodal representations, and a prediction submodel generates conditional survival probabilities for follow-up time intervals spanning several decades. MultiSurv is the first non-linear and non-proportional survival prediction method that leverages multimodal data. In addition, MultiSurv can handle missing data, including single values and complete data modalities. MultiSurv was applied to data from 33 different cancer types and yields accurate pan-cancer patient survival curves. A quantitative comparison with previous methods showed that Multisurv achieves the best results according to different time-dependent metrics. We also generated visualizations of the learned multimodal representation of MultiSurv, which revealed insights on cancer characteristics and heterogeneity.
“…Notable examples include methods designed to use clinical and gene expression data, namely DeepSurv 13 and Cox-nnet 14 . Other methods focused on imaging data, such as CXR-risk, which uses chest radiographs 15 , LungNet 16 and a gastric cancer survival prediction model 17 , which use computed tomography (CT) images, a nasopharyngeal carcinoma survival prediction model 18 , which uses magnetic resonance imaging (MRI) data, and WSISA 19 , which employs histopathology slides.…”
The age of precision medicine demands powerful computational techniques to handle high-dimensional patient data. We present MultiSurv, a multimodal deep learning method for long-term pan-cancer survival prediction. MultiSurv uses dedicated submodels to establish feature representations of clinical, imaging, and different high-dimensional omics data modalities. A data fusion layer aggregates the multimodal representations, and a prediction submodel generates conditional survival probabilities for follow-up time intervals spanning several decades. MultiSurv is the first non-linear and non-proportional survival prediction method that leverages multimodal data. In addition, MultiSurv can handle missing data, including single values and complete data modalities. MultiSurv was applied to data from 33 different cancer types and yields accurate pan-cancer patient survival curves. A quantitative comparison with previous methods showed that Multisurv achieves the best results according to different time-dependent metrics. We also generated visualizations of the learned multimodal representation of MultiSurv, which revealed insights on cancer characteristics and heterogeneity.
“…Radiomics utilizes automated quantitative characterization algorithms to transform a large number of excavatable spatial ROI-based image data into representative and effective radiomic features [ 6 ]. Recent advancements in radiomics have provided new ideas for individualized management of GC, including lymphatic metastasis prediction [ 7 , 8 ], distant metastasis prediction [ 9 ], therapeutic response evaluation [ 10 ], and prognostic evaluation [ 11 , 12 ]. These studies highlighted the value of radiomics, suggesting that radiomics could be a potential tool for the Lauren classification in GC.…”
Background
Preoperative prediction of the Lauren classification in gastric cancer (GC) is very important to the choice of therapy, the evaluation of prognosis, and the improvement of quality of life. However, there is not yet radiomics analysis concerning the prediction of Lauren classification straightly. In this study, a radiomic nomogram was developed to preoperatively differentiate Lauren diffuse type from intestinal type in GC.
Methods
A total of 539 GC patients were enrolled in this study and later randomly allocated to two cohorts at a 7:3 ratio for training and validation. Two sets of radiomic features were derived from tumor regions and peritumor regions on venous phase computed tomography (CT) images, respectively. With the least absolute shrinkage and selection operator logistic regression, a combined radiomic signature was constructed. Also, a tumor-based model and a peripheral ring-based model were built for comparison. Afterwards, a radiomic nomogram integrating the combined radiomic signature and clinical characteristics was developed. All the models were evaluated regarding classification ability and clinical usefulness.
Results
The combined radiomic signature achieved an area under receiver operating characteristic curve (AUC) of 0.715 (95% confidence interval [CI], 0.663–0.767) in the training cohort and 0.714 (95% CI, 0.636–0.792) in the validation cohort. The radiomic nomogram incorporating the combined radiomic signature, age, CT T stage, and CT N stage outperformed the other models with a training AUC of 0.745 (95% CI, 0.696–0.795) and a validation AUC of 0.758 (95% CI, 0.685–0.831). The significantly improved sensitivity of radiomic nomogram (0.765 and 0.793) indicated better identification of diffuse type GC patients. Further, calibration curves and decision curves demonstrated its great model fitness and clinical usefulness.
Conclusions
The radiomic nomogram involving the combined radiomic signature and clinical characteristics holds potential in differentiating Lauren diffuse type from intestinal type for reasonable clinical treatment strategy.
“…If an image has one or more contours associated with it, the same transformation is applied to the contours. Geometric transformations are so common that they were utilised by 92 of the 93 basic augmentation studies 15–106 …”
Section: Methodsmentioning
confidence: 99%
“…Patches are generated from the under‐represented class to even the balance. 30 articles made use of cropping 15–17,19,23,32,33,35,41,43,48,50,58,59,66,68,69,71,74,78,80,82,84,85,90,97,98,102,104,105 …”
Summary
Research in artificial intelligence for radiology and radiotherapy has recently become increasingly reliant on the use of deep learning‐based algorithms. While the performance of the models which these algorithms produce can significantly outperform more traditional machine learning methods, they do rely on larger datasets being available for training. To address this issue, data augmentation has become a popular method for increasing the size of a training dataset, particularly in fields where large datasets aren’t typically available, which is often the case when working with medical images. Data augmentation aims to generate additional data which is used to train the model and has been shown to improve performance when validated on a separate unseen dataset. This approach has become commonplace so to help understand the types of data augmentation techniques used in state‐of‐the‐art deep learning models, we conducted a systematic review of the literature where data augmentation was utilised on medical images (limited to CT and MRI) to train a deep learning model. Articles were categorised into basic, deformable, deep learning or other data augmentation techniques. As artificial intelligence models trained using augmented data make their way into the clinic, this review aims to give an insight to these techniques and confidence in the validity of the models produced.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.