Machine Learning–Assisted Recurrence Prediction for Patients With Early-Stage Non–Small-Cell Lung Cancer

Abstract

PURPOSE Stratifying patients with cancer according to risk of relapse can personalize their care. In this work, we provide an answer to the following research question: How to use machine learning to estimate probability of relapse in patients with early-stage non–small-cell lung cancer (NSCLC)? MATERIALS AND METHODS For predicting relapse in 1,387 patients with early-stage (I-II) NSCLC from the Spanish Lung Cancer Group data (average age 65.7 years, female 24.8%, male 75.2%), we train tabular and graph machine learning models. We generate automatic explanations for the predictions of such models. For models trained on tabular data, we adopt SHapley Additive exPlanations local explanations to gauge how each patient feature contributes to the predicted outcome. We explain graph machine learning predictions with an example-based method that highlights influential past patients. RESULTS Machine learning models trained on tabular data exhibit a 76% accuracy for the random forest model at predicting relapse evaluated with a 10-fold cross-validation (the model was trained 10 times with different independent sets of patients in test, train, and validation sets, and the reported metrics are averaged over these 10 test sets). Graph machine learning reaches 68% accuracy over a held-out test set of 200 patients, calibrated on a held-out set of 100 patients. CONCLUSION Our results show that machine learning models trained on tabular and graph data can enable objective, personalized, and reproducible prediction of relapse and, therefore, disease outcome in patients with early-stage NSCLC. With further prospective and multisite validation, and additional radiological and molecular data, this prognostic model could potentially serve as a predictive decision support tool for deciding the use of adjuvant treatments in early-stage lung cancer.

Examining explainable clinical decision support systems with think aloud protocols

“…From a dataset of 15,337 cancer patients, we identified a cohort of 1,348 early stage non-small-cell lung cancer patients following criteria provided by medical experts to predict risk of tumor recurrence. Patients’ data included demographics features, diagnosis features, symptoms, comorbidities, smoking information, and treatment received (surgery, chemotherapy, radiotherapy)–see Table 3 [ 34 ]. Pre-processed data was then modeled as a Knowledge Graph [ 35 ], on which we trained a graph representation learning model [ 36 , 37 ].…”

“…Predictions were obtained from the graph representation learning model (ComplEx-N3) [ 39 , 40 ]. The model achieves 68% accuracy on a 200-patient held-out test set [ 34 ]. A mock user interface rendering was developed for each of the two test patient cases.…”

Therapeutic Decision Making in Prevascular Mediastinal Tumors Using CT Radiomics and Clinical Features: Upfront Surgery or Pretreatment Needle Biopsy?

“…Three different approaches were used to develop ML classification models to discriminate between Category 1 patients and Category 2 patients. First, all clinical variables and radiomic features were used to train and validate 14 different ML models using 10-fold cross validation, a common approach for evaluating the performance of ML prediction models [35][36][37]. The dataset was divided into 10 subsets.…”

Recommendations from the Medical Education Editor

“…A series of clinical and molecular factors can be helpful; however, individualizing risk for a particular patient remains challenging 1 . Machine learning and artificial intelligence models show promise, although more work is needed to understand their utility and role 2,3 …”