Comparison of supervised machine learning classification techniques in prediction of locoregional recurrences in early oral tongue cancer

Abstract

Background:The proper estimate of the risk of recurrences in early-stage oral tongue squamous cell carcinoma (OTSCC) is mandatory for individual treatment-decision making. However, this remains a challenge even for experienced multidisciplinary centers. Objectives:We compared the performance of four machine learning (ML) algorithms for predicting the risk of locoregional recurrences in patients with OTSCC. These algorithms were Support Vector Machine (SVM), Naive Bayes (NB), Boosted Decision Tree (BDT), and Decision Forest (DF). Materials and methods:The study cohort comprised 311 cases from the five University Hospitals in Finland and A.C. Camargo Cancer Center, São Paulo, Brazil. For comparison of the algorithms, we used the harmonic mean of precision and recall called F1 score, specificity, and accuracy values. These algorithms and their corresponding permutation feature importance (PFI) with the input parameters were externally tested on 59 new cases. Furthermore, we compared the performance of the algorithm that showed the highest prediction accuracy with the prognostic significance of depth of invasion (DOI). Results:The results showed that the average specificity of all the algorithms was 71%. The SVM showed an accuracy of 68% and F1 score of 0.63, NB an accuracy of 70% and F1 score of 0.64, BDT an accuracy of 81% and F1 score of 0.78, and DF an accuracy of 78% and F1 score of 0.70. Additionally, these algorithms outperformed the DOI-based approach, which gave an accuracy of 63%. With PFIanalysis, there was no significant difference in the overall accuracies of three of the algorithms; PFI-BDT accuracy increased to 83.1%, PFI-DF increased to 80%, PFI-SVM decreased to 64.4%, while PFI-NB accuracy increased significantly to 81.4%. Conclusions: Our findings show that the best classification accuracy was achieved with the boosted decision tree algorithm. Additionally, these algorithms outperformed the DOI-based approach. Furthermore, with few parameters identified in the PFI analysis, ML technique still showed the ability to predict locoregional recurrence. The application of boosted decision tree machine learning algorithm can stratify OTSCC patients and thus aid in their individual treatment planning.

Machine learning and treatment outcome prediction for oral cancer

“…Their popularity is based upon a presumed ability to sequentially detect patterns, garner information and undergo automated training based on data input, especially complex non‐homogenous data, ultimately making clinical predictions with minimal human intervention 13,14 . Whilst a degree of predictive accuracy for algorithms has been reported, in particular the use of support vector machines, boosted decision trees, decision forest and artificial neural networks, there is a need to validate the predictive power of machine learning by analysing disease progression within well‐defined OSCC patient cohorts prior to widespread translation to clinical practice 15‐19 …”

Burns Depth Assessment Using Deep Learning Features

“…In order to evaluate the performance of the prediction, there is need to interpret the values obtianed in the Tables 3 and 4. These evaluation measuresare based on the following parameters: True Positives (TP), True Negatives (TN), False Positives (FP) and False Negatives (FN) [38,39]. Accuracy is on of those metrices that gives the general performance of the classification.…”

Survival prediction for bladder cancer using machine learning: development of BlaCaSurv online survival prediction application

“…Support vector machine (SVM) uses a hyperplane to separate the two classes of the output. 17 The algorithm tries to maximize the distance between the hyperplane and the two closest data points from each class.…”

“…11 With respect to cancer research a growing body of literature has shown application of machine learning for predicting cancer survival from hospital records and registries. [12][13][14][15][16][17] The largest publicly available source of cancer statistics in the United States is the Surveillance Epidemiology and End Result (SEER) with a representation of 28% of the population. 18 Using SEER database, a few studies have applied machine learning to predict patient survival on various cancers.…”