Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI

Abstract

he prospect of improved clinical outcomes and more efficient health systems has fueled a rapid rise in the development and evaluation of AI systems over the last decade. Because most AI systems within healthcare are complex interventions designed as clinical decision support systems, rather than autonomous agents, the interactions among the AI systems, their users and the implementation environments are defining components of the AI interventions' overall potential effectiveness. Therefore, bringing AI systems from mathematical performance to clinical utility needs an adapted, stepwise implementation and evaluation pathway, addressing the complexity of this collaboration between two independent forms of intelligence, beyond measures of effectiveness alone 1 . Despite indications that some AI-based algorithms now match the accuracy of human experts within preclinical in silico studies 2 , there

A systematic review of radiomics in pancreatitis: applying the evidence level rating tool for promoting clinical transferability

“…During this stage, the researchers, authors, reviewers, and editors of radiomics studies play an important role to improve the methodological and reporting quality, and make sure only studies with adequate innovation are being published. Next, at the stage of safety and utility, the small-scale early live clinical evaluations are used to inform regulatory decisions and are part of the clinical evidence generation process [ 65 ]. With improvements of study quality, the radiomics research community could for the first time provide more robust scientific evidence for the translation of radiomics.…”

“…Indeed, the IBSI guideline used in our review is an achievement gained by an independent international collaboration which works towards standardization of the radiomics methodology and reporting [ 11 ]. There are many other guidelines developed or under development by the radiomics and artificial intelligence community with the purpose to improve study quality, including Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis based on Artificial Intelligence (TRIPOD-AI) [ 63 ], Prediction model Risk Of Bias ASsessment Tool based on Artificial Intelligence (PROBAST-AI) [ 63 ], Quality Assessment of Diagnostic Accuracy Studies centered on Artificial Intelligence (QUADAS-AI) [ 64 ], Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI) [ 65 ], Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence (SPRIIT-AI) [ 66 ], Consolidated Standards of Reporting Trials–Artificial Intelligence (CONSORT-AI) [ 67 ], Standards for Reporting of Diagnostic Accuracy Study centered on Artificial Intelligence (STARD-AI) [ 68 ], Checklist for Artificial Intelligence in Medical Imaging (CLAIM) [ 69 ], etc. Their project teams and steering committees usually consisted of a broad range of experts to provide balanced and diverse views involving various stakeholder groups.…”

Bias in Machine Learning Models Can Be Significantly Mitigated by Careful Training: Evidence from Neuroimaging Studies

“…As machine learning models are being applied to diverse problems in the clinical sciences, there is an increasing amount of discussion on bias in particular, and ethical issues in general 21,23,[33][34][35] . This literature is playing a crucial role in shaping policies for deployment of automated diagnostic models.…”

Artificial Intelligence in Colorectal Cancer Surgery: Present and Future Perspectives

“…To conclude, well designed implementation studies looking at how to integrate such technology in complex clinical and surgical workflows and assessing how these changes impact patient care are crucial to measure actual value for patients and healthcare systems. Translational studies exploring the clinical value of surgical AI still remain to be published, but currently available guidelines can help designing protocols [ 37 ], early assessments [ 38 ], and reporting of AI-based interventions [ 39 ].…”