Empirical comparison of routinely collected electronic health record data for head and neck cancer‐specific survival in machine‐learnt prognostic models

Abstract

Background Knowledge of the prognostic factors and performance of machine learning predictive models for 2‐year cancer‐specific survival (CSS) is limited in the head and neck cancer (HNC) population. Methods Data from our facilities' oncology information system (OIS) collected for routine practice (OIS dataset, n = 430 patients) and research purposes (research dataset, n = 529 patients) were extracted on adults diagnosed between 2000 and 2017 with squamous cell carcinoma of the head and neck. Results Machine learning demonstrated excellent performance (area under the curve, AUC) in the whole cohort (AUC = 0.97, research dataset), larynx cohort (AUC = 0.98, both datasets), and oropharynx cohort (AUC = 0.99, both datasets). Tumor site and T classification were identified as predictors of 2‐year CSS in both datasets. Hypothyroidism and fitness for operation were further identified in the research dataset. Conclusions Datasets extracted from an OIS for routine clinical practice and research purposes demonstrated high utility for informing 2‐year head and neck CSS.

Prognosing post-treatment outcomes of head and neck cancer using structured data and machine learning: A systematic review

“…For the oropharynx, three studies reported C-indices ranging from 0.77 to 0.80, [ 72 , 73 , 75 ], but there was not enough data to consolidate the C-index for Cox PH models. For the hypopharynx and nasopharynx, only two studies reported C-indices for ML models, ranging from 0.72 to 0.79 [ 72 , 75 ] and from 0.72 to 0.83 [ 76 , 77 ]. The C-index for Cox PH could not be consolidated for hypopharynx and nasopharynx.…”

“…For HNC as a whole, four studies reported AUROC ranging from 0.75 to 0.97, and three of these studies also reported F1-scores ranging from 0.65 to 0.89; based on three of these studies, AUROC for logistic regressions ranged from 0.71 to 0.84, and based on two studies, F1-scores ranged from 0.54 to 0.77 [ 67 , 68 , 75 , 76 ]. For the oral cavity, seven studies reported AUROC and F1-score for ML models, ranging from 0.61 to 0.91 and 0.58 to 0.86, respectively, with four of these studies also reporting on logistic regression models, which ranged from 0.52 to 0.69 for AUROC and 0.57 to 0.62 for F1-score [ 56 , 58 , 63 , 75 , 76 , 87 , 88 ]. For the larynx, four studies reported AUROC for ML models ranging from 0.76 to 0.97, and three of these studies also reported F1-scores ranging from 0.63 to 0.92; from these four, two studies also reported on logistic regression with AUROC ranging from 0.76 to 0.92 [ 75 , 76 , 81 , 85 ].…”

“…For the oral cavity, seven studies reported AUROC and F1-score for ML models, ranging from 0.61 to 0.91 and 0.58 to 0.86, respectively, with four of these studies also reporting on logistic regression models, which ranged from 0.52 to 0.69 for AUROC and 0.57 to 0.62 for F1-score [ 56 , 58 , 63 , 75 , 76 , 87 , 88 ]. For the larynx, four studies reported AUROC for ML models ranging from 0.76 to 0.97, and three of these studies also reported F1-scores ranging from 0.63 to 0.92; from these four, two studies also reported on logistic regression with AUROC ranging from 0.76 to 0.92 [ 75 , 76 , 81 , 85 ]. For the oropharynx, three studies reported AUROC for ML models ranging from 0.93 to 0.97 and F1-scores ranging from 0.90 to 0.92, but there was not enough information to consolidate results for logistic regression [ 74 – 76 ].…”

“…For the larynx, four studies reported AUROC for ML models ranging from 0.76 to 0.97, and three of these studies also reported F1-scores ranging from 0.63 to 0.92; from these four, two studies also reported on logistic regression with AUROC ranging from 0.76 to 0.92 [ 75 , 76 , 81 , 85 ]. For the oropharynx, three studies reported AUROC for ML models ranging from 0.93 to 0.97 and F1-scores ranging from 0.90 to 0.92, but there was not enough information to consolidate results for logistic regression [ 74 – 76 ]. Moreover, the AUROC of ML models ranged from 0.77 to 0.85 for two studies that reported on the hypopharynx [ 75 , 82 ].…”

“…Among ML models, tree-based models were used more often and generally demonstrated superior performance in terms of AUROC and F1-scores compared to other algorithms. In nine studies that conducted comparative analyses, six found that tree-based models, including random forest and XGBoost, outperformed others [ 58 , 63 , 76 , 82 , 85 , 87 ]. However, in one study, a voting classifier of random forest, logistic regression, and Gaussian Naïve Bayes was superior [ 88 ].…”