Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative

Spadarella, Gaia; Stanzione, Arnaldo; D’Antonoli, Tugba Akinci; Andreychenko, Anna; Fanni, Salvatore Claudio; Ugga, Lorenzo; Kotter, Elmar; Cuocolo, Renato

doi:10.1007/s00330-022-09187-3

Cited by 59 publications

(36 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The mean RQS was 11.34 ± 3.68, resulting in 31.27 ± 10.40%. This result is higher than the average results, with an RQS score of 18.87% reported in a systematic review of RQS applications published by the EuSoMII Radiomics Auditing Group [ 39 ]. This difference may be largely explained by the limits of the RQS tool itself.…”

Section: Discussionmentioning

confidence: 56%

The Role of Radiomics in Salivary Gland Imaging: A Systematic Review and Radiomics Quality Assessment

et al. 2022

Self Cite

View full text Add to dashboard Cite

Background: Radiomics of salivary gland imaging can support clinical decisions in different clinical scenarios, such as tumors, radiation-induced xerostomia and sialadenitis. This review aims to evaluate the methodological quality of radiomics studies on salivary gland imaging. Material and Methods: A systematic search was performed, and the methodological quality was evaluated using the radiomics quality score (RQS). Subgroup analyses according to the first author’s professional role (medical or not medical), journal type (radiological journal or other) and the year of publication (2021 or before) were performed. The correlation of RQS with the number of patients was calculated. Results: Twenty-three articles were included (mean RQS 11.34 ± 3.68). Most studies well-documented the imaging protocol (87%), while neither prospective validations nor cost-effectiveness analyses were performed. None of the included studies provided open-source data. A statistically significant difference in RQS according to the year of publication was found (p = 0.009), with papers published in 2021 having slightly higher RQSs than older ones. No differences according to journal type or the first author’s professional role were demonstrated. A moderate relationship between the overall RQS and the number of patients was found. Conclusions: Radiomics application in salivary gland imaging is increasing. Although its current clinical applicability can be affected by the somewhat inadequate quality of the papers, a significant improvement in radiomics methodologies has been demonstrated in the last year.

show abstract

Section: Discussionmentioning

confidence: 56%

The Role of Radiomics in Salivary Gland Imaging: A Systematic Review and Radiomics Quality Assessment

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sollini et al [ 16 ] declared that the number of oncological image minding studies was six-times of those in non-oncological field. Spadarella et al [ 17 ] found that more than nine tenths of their included systematic reviews focused on oncological radiomics. It is not surprising because the concept of radiomics was raised to mine the medical images for extra deeper information related to oncological genomics [ 1 ].…”

Section: Discussionmentioning

confidence: 99%

“…Although an increasing number of systematic reviews and meta-analyses are published in various medical fields, including radiomics [ 15 ], it is still unclear how far radiomics is from current research to clinical application [ 9 – 13 ]. There were systematic reviews attempt to cover a wide range of topics in radiomics [ 16 , 17 ]. However, the number of published primary radiomics studies was too large to summarize in one single systematic review [ 16 ], and the evidence level rating of current radiomics was out of the aim of a methodological systematic review [ 17 ].…”

Section: Introductionmentioning

confidence: 99%

“…There were systematic reviews attempt to cover a wide range of topics in radiomics [ 16 , 17 ]. However, the number of published primary radiomics studies was too large to summarize in one single systematic review [ 16 ], and the evidence level rating of current radiomics was out of the aim of a methodological systematic review [ 17 ]. Nevertheless, the overview of systematic reviews is a relatively new type of publication that attempts to provide a broader evidence synthesis highlighting the knowledge gaps, biases, and priorities for future research, which helps clinical practitioners and policy-makers interpret the results of higher-level pieces of evidence in radiomics [ 18 – 20 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An overview of meta-analyses on radiomics: more evidence is needed to support clinical translation

Zhong

Zhang

et al. 2023

Insights Imaging

View full text Add to dashboard Cite

Objective To conduct an overview of meta-analyses of radiomics studies assessing their study quality and evidence level. Methods A systematical search was updated via peer-reviewed electronic databases, preprint servers, and systematic review protocol registers until 15 November 2022. Systematic reviews with meta-analysis of primary radiomics studies were included. Their reporting transparency, methodological quality, and risk of bias were assessed by PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) 2020 checklist, AMSTAR-2 (A MeaSurement Tool to Assess systematic Reviews, version 2) tool, and ROBIS (Risk Of Bias In Systematic reviews) tool, respectively. The evidence level supporting the radiomics for clinical use was rated. Results We identified 44 systematic reviews with meta-analyses on radiomics research. The mean ± standard deviation of PRISMA adherence rate was 65 ± 9%. The AMSTAR-2 tool rated 5 and 39 systematic reviews as low and critically low confidence, respectively. The ROBIS assessment resulted low, unclear and high risk in 5, 11, and 28 systematic reviews, respectively. We reperformed 53 meta-analyses in 38 included systematic reviews. There were 3, 7, and 43 meta-analyses rated as convincing, highly suggestive, and weak levels of evidence, respectively. The convincing level of evidence was rated in (1) T2-FLAIR radiomics for IDH-mutant vs IDH-wide type differentiation in low-grade glioma, (2) CT radiomics for COVID-19 vs other viral pneumonia differentiation, and (3) MRI radiomics for high-grade glioma vs brain metastasis differentiation. Conclusions The systematic reviews on radiomics were with suboptimal quality. A limited number of radiomics approaches were supported by convincing level of evidence. Clinical relevance statement The evidence supporting the clinical application of radiomics are insufficient, calling for researches translating radiomics from an academic tool to a practicable adjunct towards clinical deployment. Graphical Abstract

show abstract

“… 100 However, the exponential growth in the number of publications and commercial products has not been matched by an equal increase in the quality or transparency of study methodology. 101 , 102 This is supported by a recent survey of all systematic review papers using the Radiomics Quality Score to assess methodological quality and transparency in medical imaging. Of the 44 included articles, each evaluating an average of 32 research papers, the median score was 21%, with a stable trend over the years (ranging from 2018 to 2021).…”

Section: Must Have Qualitiesmentioning

confidence: 91%

Must-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning

Koçak¹,

Cuocolo²,

Santos³

et al. 2023

Balkan Med J

Self Cite

View full text Add to dashboard Cite

In the field of computer science, known as artificial intelligence, algorithms imitate reasoning tasks that are typically performed by humans. The techniques that allow machines to learn and get better at tasks such as recognition and prediction, which form the basis of clinical practice, are referred to as machine learning, which is a subfield of artificial intelligence. The number of artificial intelligence-and machine learnings-related publications in clinical journals has grown exponentially, driven by recent developments in computation and the accessibility of simple tools. However, clinicians are often not included in data science teams, which may limit the clinical relevance, explanability, workflow compatibility, and quality improvement of artificial intelligence solutions. Thus, this results in the language barrier between clinicians and artificial intelligence developers. Healthcare practitioners sometimes lack a basic understanding of artificial intelligence research because the approach is difficult for non-specialists to understand. Furthermore, many editors and reviewers of medical publications might not be familiar with the fundamental ideas behind these technologies, which may prevent journals from publishing high-quality artificial intelligence studies or, worse still, could allow for the publication of low-quality works. In this review, we aim to improve readers’ artificial intelligence literacy and critical thinking. As a result, we concentrated on what we consider the 10 most important qualities of artificial intelligence research: valid scientific purpose, high-quality data set, robust reference standard, robust input, no information leakage, optimal bias-variance tradeoff, proper model evaluation, proven clinical utility, transparent reporting, and open science. Before designing a study, one should have defined a sound scientific purpose. Then, it should be backed by a high-quality data set, robust input, and a solid reference standard. The artificial intelligence development pipeline should prevent information leakage. For the models, optimal bias-variance tradeoff should be achieved, and generalizability assessment must be adequately performed. The clinical value of the final models must also be established. After the study, thought should be given to transparency in publishing the process and results as well as open science for sharing data, code, and models. We hope this work may improve the artificial intelligence literacy and mindset of the readers.

show abstract

Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative

Cited by 59 publications

References 72 publications

The Role of Radiomics in Salivary Gland Imaging: A Systematic Review and Radiomics Quality Assessment

The Role of Radiomics in Salivary Gland Imaging: A Systematic Review and Radiomics Quality Assessment

An overview of meta-analyses on radiomics: more evidence is needed to support clinical translation

Must-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning

Contact Info

Product

Resources

About