Imaging Biomarkers of Glioblastoma Treatment Response: A Systematic Review and Meta-Analysis of Recent Machine Learning Studies

Abstract

ObjectiveMonitoring biomarkers using machine learning (ML) may determine glioblastoma treatment response. We systematically reviewed quality and performance accuracy of recently published studies.MethodsFollowing Preferred Reporting Items for Systematic Reviews and Meta-Analysis: Diagnostic Test Accuracy, we extracted articles from MEDLINE, EMBASE and Cochrane Register between 09/2018–01/2021. Included study participants were adults with glioblastoma having undergone standard treatment (maximal resection, radiotherapy with concomitant and adjuvant temozolomide), and follow-up imaging to determine treatment response status (specifically, distinguishing progression/recurrence from progression/recurrence mimics, the target condition). Using Quality Assessment of Diagnostic Accuracy Studies Two/Checklist for Artificial Intelligence in Medical Imaging, we assessed bias risk and applicability concerns. We determined test set performance accuracy (sensitivity, specificity, precision, F1-score, balanced accuracy). We used a bivariate random-effect model to determine pooled sensitivity, specificity, area-under the receiver operator characteristic curve (ROC-AUC). Pooled measures of balanced accuracy, positive/negative likelihood ratios (PLR/NLR) and diagnostic odds ratio (DOR) were calculated. PROSPERO registered (CRD42021261965).ResultsEighteen studies were included (1335/384 patients for training/testing respectively). Small patient numbers, high bias risk, applicability concerns (particularly confounding in reference standard and patient selection) and low level of evidence, allow limited conclusions from studies. Ten studies (10/18, 56%) included in meta-analysis gave 0.769 (0.649-0.858) sensitivity [pooled (95% CI)]; 0.648 (0.749-0.532) specificity; 0.706 (0.623-0.779) balanced accuracy; 2.220 (1.560-3.140) PLR; 0.366 (0.213-0.572) NLR; 6.670 (2.800-13.500) DOR; 0.765 ROC-AUC.ConclusionML models using MRI features to distinguish between progression and mimics appear to demonstrate good diagnostic performance. However, study quality and design require improvement.

High-Grade Glioma Treatment Response Monitoring Biomarkers: A Position Statement on the Evidence Supporting the Use of Advanced MRI Techniques in the Clinic, and the Latest Bench-to-Bedside Developments. Part 2: Spectroscopy, Chemical Exchange Saturation, Multiparametric Imaging, and Radiomics

“…Many incorporate machine learning as a central pillar of the process. A review of studies up to 2018 ( 91 ), a systematic review of studies from 2018 – 2020 ( 122 ) using PRISMA-DTA methodology and a meta-analysis from 2018–2021 ( 123 ) indicated that those taking advantage of enhanced computational processing power to build monitoring biomarker models (e.g., using deep learning methods such as convolutional neural networks) have yet to show an advantage in performance compared with machine learning techniques using explicit feature engineering and less computationally expensive classifiers (e.g., using “classical” machine learning methods support vector machine). It is also notable that studies applying machine learning to build monitoring biomarker models have yet to show an overall advantage over those using traditional statistical methods.…”

“…It is also notable that studies applying machine learning to build monitoring biomarker models have yet to show an overall advantage over those using traditional statistical methods. There is good diagnostic performance of machine learning models that use MRI features to distinguish between progressive disease and diagnostic accuracy measures comprise recall = 0.61 – 1.00, specificity = 0.47 – 0.90, balanced accuracy = 0.54 – 0.83, precision = 0.58 – 0.88, F1 score = 0.59 – 0.94, and AUC = 0.65 – 0.85 ( 122 , 123 ). The recent meta-analysis of ten studies showed a pooled true positive rate (sensitivity) = 0.769 (0.649 – 0.858), a false positive rate (1-specificity) = 0.352 (0.251 – 0.468) and a summary AUC-ROC = 0.765.…”

“…The recent meta-analysis of ten studies showed a pooled true positive rate (sensitivity) = 0.769 (0.649 – 0.858), a false positive rate (1-specificity) = 0.352 (0.251 – 0.468) and a summary AUC-ROC = 0.765. Other pooled metrics showed derived measures of balanced accuracy = 0.706 (0.623–0.779); positive likelihood ratio = 2.220 (1.560–3.140); negative likelihood ratio = 0.366 (0.213–0.572); and diagnostic odds ratio = 6.670 (2.800–13.500) ( 123 ). It is noteworthy that the small numbers of patients included in these studies, the high-risk of bias and concerns of applicability in the study designs, and the low level of evidence given that the monitoring biomarker studies are retrospective, suggest that limited conclusions can be drawn from the data.…”

Detection of cerebral aneurysms using artificial intelligence: a systematic review and meta-analysis

“…The first group consists of 13 studies that were published before 2018 (pre-2018), and the second group consists of 30 studies that were published in 2018 or later (post-2018). This arbitrary cut-off was chosen as it mirrors the notable observation that in 2018, arXiv (a repository where computer science papers are self-archived before publication in a peer reviewed journal) surpassed 100 new machine learning pre-prints per day 65 66. In the pre-2018 group, the lesion sensitivities ranged from 56.8% to 100% with false-positives/case ranging from 2.3 to 31.8.…”

Non-invasive tumor probability maps developed using autopsy tissue identify novel areas of tumor beyond the imaging-defined margin

“…Apparent diffusion coefficient (ADC) images derived from diffusion-weighted imaging have shown promise in highlighting areas of diffusion restricting hypercellularity associated with tumor, though recent studies validating this signature beyond the contrast-enhancing margin have disputed the strength of this relationship [14][15][16][17][18][19] . Machine learning approaches have also sought to maximize the amount of clinically relevant information we can extract from these non-invasive images, employing recent advances in computing to automatically segment the radiologist-defined margin, identify patient-level genetic signatures such as IDH1 mutation and MGMT methylation status, and predict cellular-level information using biopsy samples as validation [20][21][22][23][24] .…”