Machine-learning prediction of cancer survival: a retrospective study using electronic administrative records and a cancer registry

Abstract

ObjectivesUsing the prediction of cancer outcome as a model, we have tested the hypothesis that through analysing routinely collected digital data contained in an electronic administrative record (EAR), using machine-learning techniques, we could enhance conventional methods in predicting clinical outcomes.SettingA regional cancer centre in Australia.ParticipantsDisease-specific data from a purpose-built cancer registry (Evaluation of Cancer Outcomes (ECO)) from 869 patients were used to predict survival at 6, 12 and 24 months. The model was validated with data from a further 94 patients, and results compared to the assessment of five specialist oncologists. Machine-learning prediction using ECO data was compared with that using EAR and a model combining ECO and EAR data.Primary and secondary outcome measuresSurvival prediction accuracy in terms of the area under the receiver operating characteristic curve (AUC).ResultsThe ECO model yielded AUCs of 0.87 (95% CI 0.848 to 0.890) at 6 months, 0.796 (95% CI 0.774 to 0.823) at 12 months and 0.764 (95% CI 0.737 to 0.789) at 24 months. Each was slightly better than the performance of the clinician panel. The model performed consistently across a range of cancers, including rare cancers. Combining ECO and EAR data yielded better prediction than the ECO-based model (AUCs ranging from 0.757 to 0.997 for 6 months, AUCs from 0.689 to 0.988 for 12 months and AUCs from 0.713 to 0.973 for 24 months). The best prediction was for genitourinary, head and neck, lung, skin, and upper gastrointestinal tumours.ConclusionsMachine learning applied to information from a disease-specific (cancer) database and the EAR can be used to predict clinical outcomes. Importantly, the approach described made use of digital data that is already routinely collected but underexploited by clinical health systems.

Predicting mortality over different time horizons: which data elements are needed?

“…These results correspond with previous studies using EHR data to develop risk models, illustrating that EHR-based models perform better with nearer-term events. [14][15][16][17][18][19][20][21] Moreover, when comparing the "important" variables over different time horizons, previous work has similarly suggested that more "dynamic" metrics are important for nearer-term outcomes and more "stable" metrics are important for longer-term events. 17 This finding stresses the importance of machine-learning methods capable of handling large numbers of disparate predictor variables.…”

eDoctor: machine learning and the future of medicine

“…This trend of AI models outperforming physician estimates is continued in that Gupta et al . took a pre‐existing cancer registry and electronic administrative records (not medical records) and found that it provided better outcome prediction than previously used models or a clinician panel.…”

2015 Marshall Urist Young Investigator Award: Prognostication in Patients With Long Bone Metastases: Does a Boosting Algorithm Improve Survival Estimates?

“…The average performance (the ability of a model to separate patients with different outcomes) was calculated over the five training and testing repetitions per multiple imputed dataset for all three models and prediction periods and subsequently pooled. Performance was assessed using receiver operating characteristic (ROC) curves [4,11,27]. ROC curves are made by plotting the rate of false positives (1 -specificity) on the x-axis and the rate of true positives (sensitivity) on the y-axis for all threshold values.…”