Quality of Radiomics Research on Brain Metastasis: A Roadmap to Promote Clinical Translation

Abstract

Objective Our study aimed to evaluate the quality of radiomics studies on brain metastases based on the radiomics quality score (RQS), Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist, and the Image Biomarker Standardization Initiative (IBSI) guidelines. Materials and Methods PubMed MEDLINE, and EMBASE were searched for articles on radiomics for evaluating brain metastases, published until February 2021. Of the 572 articles, 29 relevant original research articles were included and evaluated according to the RQS, TRIPOD checklist, and IBSI guidelines. Results External validation was performed in only three studies (10.3%). The median RQS was 3.0 (range, -6 to 12), with a low basic adherence rate of 50.0%. The adherence rate was low in comparison to the “gold standard” (10.3%), stating the potential clinical utility (10.3%), performing the cut-off analysis (3.4%), reporting calibration statistics (6.9%), and providing open science and data (3.4%). None of the studies involved test-retest or phantom studies, prospective studies, or cost-effectiveness analyses. The overall rate of adherence to the TRIPOD checklist was 60.3% and low for reporting title (3.4%), blind assessment of outcome (0%), description of the handling of missing data (0%), and presentation of the full prediction model (0%). The majority of studies lacked pre-processing steps, with bias-field correction, isovoxel resampling, skull stripping, and gray-level discretization performed in only six (20.7%), nine (31.0%), four (3.8%), and four (13.8%) studies, respectively. Conclusion The overall scientific and reporting quality of radiomics studies on brain metastases published during the study period was insufficient. Radiomics studies should adhere to the RQS, TRIPOD, and IBSI guidelines to facilitate the translation of radiomics into the clinical field.

An updated systematic review of radiomics in osteosarcoma: utilizing CLAIM to adapt the increasing trend of deep learning application in radiomics

“…The eligible studies employed RQS [ 7 ], TRIPOD [ 10 ], CLAIM [ 19 ], and QUADAS-2 tools [ 9 ] (Additional file 1 : Tables S2–S5). The RQS is a consensus list composed of sixteen items for methodological issues specific to radiomics studies and is later summarized to six key domains [ 12 – 14 ]. The TRIPOD statement provides a checklist consisting of thirty-seven items in twenty-two criteria aiming to promote transparency of prediction model studies and is recommended for identifying room of improvement in radiomics studies [ 12 – 14 ].…”

“…The RQS is a consensus list composed of sixteen items for methodological issues specific to radiomics studies and is later summarized to six key domains [ 12 – 14 ]. The TRIPOD statement provides a checklist consisting of thirty-seven items in twenty-two criteria aiming to promote transparency of prediction model studies and is recommended for identifying room of improvement in radiomics studies [ 12 – 14 ]. The CLAIM includes forty-two items in seven topics that should be viewed as a best practice to guide presentation of AI research [ 19 ].…”

“…The RQS rating, the ideal percentage of RQS, and adherence rates of RQS, TRIPOD and CLIAM were calculated. In case a score of at least one point for each item was obtained without minus points, it was considered to have basic adherence, as those have been reported [ 12 – 14 ]. For example, if the item of validation in RQS obtained 2–5 points, it was considered as basic adherent, while it was regarded as without basic adherence when it was rated as -5 points.…”

Artificial Intelligence in Neuro-Oncologic Imaging: A Brief Review for Clinical Use Cases and Future Perspectives

“…In brain metastasis, radiomics applied in prediction of the primary tumor types [ 17 ] or prediction of EGFR mutation status in non-small cell lung cancer [ 18 ]. Due to heterogeneous nature of patients cohort and small size of the lesions, the quality of radiogenomics research on brain metastasis is not often satisfactory [ 19 ]. In deep learning, CNN is the most commonly used algorithm.…”

Guides for the Successful Conduct and Reporting of Systematic Review and Meta-Analysis of Diagnostic Test Accuracy Studies

“…These specific guides should be referred to appropriately. Examples elucidating RQS use can be found elsewhere [ 14 15 ].…”