Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Sounderajah, Viknesh; Ashrafian, Hutan; Golub, Robert M.; Shetty, Shravya; Fauw, Jeffrey De; Hooft, Lotty; Moons, Karel G.M.; Collins, Gary S.; Moher, David; Bossuyt, Patrick M.; Darzi, Ara; Karthikesalingam, Alan; Denniston, Alastair K.O.; Mateen, Bilal A.; Ting, Daniel Shu Wei; Treanor, Darren; King, Dominic; Greaves, Felix; Godwin, Jonathan; Pearson‐Stuttard, Jonathan; Harling, Leanne; McInnes, Matthew D. F.; Rifai, Nader; Tomašev, Nenad; Normahani, Pasha; Whiting, Penny; Aggarwal, Ravi; Vollmer, Sebastian J.; Markar, Sheraz R.; Panch, Trishan; Liu, Xiaoxuan

doi:10.1136/bmjopen-2020-047709

Cited by 127 publications

(69 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To date, the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015 statement remains the most used tool for reporting of studies investigating diagnostic test accuracy and performance [ 43 , 44 ]. However, this tool has some shortcomings when reporting studies evaluating artificial intelligence (AI) driven interventions due to unclear methodological interpretation, lack of standardised nomenclature, use of unfamiliar outcome measures, and other issues, thereby limiting the comprehensive appraisal of these technologies [ 45 ]. Thus, our study findings further reiterate the need for developing an AI-specific STARD guideline to ensure complete and robust reporting of studies evaluating AI-driven technologies and interventions [ 45 , 46 ].…”

Section: Discussionmentioning

confidence: 99%

Digital auscultation as a diagnostic aid to detect childhood pneumonia: A systematic review

et al. 2022

View full text Add to dashboard Cite

Background Frontline health care workers use World Health Organization Integrated Management of Childhood Illnesses (IMCI) guidelines for child pneumonia care in low-resource settings. IMCI guideline pneumonia diagnostic criterion performs with low specificity, resulting in antibiotic overtreatment. Digital auscultation with automated lung sound analysis may improve the diagnostic performance of IMCI pneumonia guidelines. This systematic review aims to summarize the evidence on detecting adventitious lung sounds by digital auscultation with automated analysis compared to reference physician acoustic analysis for child pneumonia diagnosis. Methods In this review, articles were searched from MEDLINE, Embase, CINAHL Plus, Web of Science, Global Health, IEEExplore database, Scopus, and the ClinicalTrial.gov databases from the inception of each database to October 27, 2021, and reference lists of selected studies and relevant review articles were searched manually. Studies reporting diagnostic performance of digital auscultation and/or computerized lung sound analysis compared against physicians’ acoustic analysis for pneumonia diagnosis in children under the age of 5 were eligible for this systematic review. Retrieved citations were screened and eligible studies were included for extraction. Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. All these steps were independently performed by two authors and disagreements between the reviewers were resolved through discussion with an arbiter. Narrative data synthesis was performed. Results A total of 3801 citations were screened and 46 full-text articles were assessed. 10 studies met the inclusion criteria. Half of the studies used a publicly available respiratory sound database to evaluate their proposed work. Reported methodologies/approaches and performance metrics for classifying adventitious lung sounds varied widely across the included studies. All included studies except one reported overall diagnostic performance of the digital auscultation/computerised sound analysis to distinguish adventitious lung sounds, irrespective of the disease condition or age of the participants. The reported accuracies for classifying adventitious lung sounds in the included studies varied from 66.3% to 100%. However, it remained unclear to what extent these results would be applicable for classifying adventitious lung sounds in children with pneumonia. Conclusions This systematic review found very limited evidence on the diagnostic performance of digital auscultation to diagnose pneumonia in children. Well-designed studies and robust reporting are required to evaluate the accuracy of digital auscultation in the paediatric population.

show abstract

Section: Discussionmentioning

confidence: 99%

Digital auscultation as a diagnostic aid to detect childhood pneumonia: A systematic review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Of particular relevance to diagnostic radiologists is the work to develop STARD‐AI (Standards for Reporting of Diagnostic Accuracy Studies—Artificial Intelligence), an extension of the original STARD Statement, which was updated in 2016. Sounderajah 21 et al . state, in their paper, describing the STARD‐AI methodology: ‘……much of the evidence supporting diagnostic algorithms has been disseminated in the absence of AI‐specific reporting guidelines’.…”

Section: Design and Reporting Of Ai / ML Studies: Standardisation To ...mentioning

confidence: 99%

Quality use of artificial intelligence in medical imaging: What do radiologists need to know?

2022

View full text Add to dashboard Cite

The application of artificial intelligence, and in particular machine learning, to the practice of radiology, is already impacting the quality of imaging care. It will increasingly do so in the future. Radiologists need to be aware of factors that govern the quality of these tools at the development, regulatory and clinical implementation stages in order to make judicious decisions about their use in daily practice.

show abstract

“…Additional guidelines for AI clinical research quality assessment include, QUADAS-AI, which is an AI-centered diagnostic test accuracy quality assessment tool (27); STARD-AI, which guides AI-centered diagnostic test accuracy studies (28); and DECIDE-AI, which is a reporting guideline to bridge the development-to-implementation gap in clinical AI (29). New quality metrics and guidelines will be developed as AI applications continue to expand.…”

Section: Guidelines For ML Dl and Ai Clinical Trials Design And Repor...mentioning

confidence: 99%

Clinical Machine Learning Modeling Studies: Methodology and Data Reporting

Dumitrascu

Wang

Chen

2022

Journal of Neuro-Ophthalmology

View full text Add to dashboard Cite

A rtificial intelligence (AI) is a branch of computer science dealing with independent intelligent behavior in computers (1), aiming to develop computer systems that can perform tasks normally requiring human intelligence (2). AI is a tool with the potential to automate medical data interpretation to facilitate early diagnosis and timely management of disease, with image interpretation being of particular interest (3-5). Such approaches may increase efficiency and capacity in neuroophthalmology, which is facing a current and prospective shortage of providers (6). Machine learning (ML) is the process through which the intelligence is created. Deep learning (DL) is a subtype of ML that uses convolutional neural networks (CNN). Driven by the potential medical applications, advances in computer science and computing power, ML and AI studies in ophthalmology have recently increased, comprising multiple computer science strategies, with deep learning (DL) being the most advanced methodology to date (7). Relevant to neuro-ophthalmology is the recent study by Liu et al ( 8) which used DL to develop a deep CNN to classify a spectrum of optic disc abnormalities in color fundus photographs. The ultimate AI aim of this study was to determine whether the CNN could automatically distinguish between normal and abnormal optic discs. Similar forthcoming studies and trials in ophthalmology and neuro-ophthalmology make critical appraisal of ML/AI-based studies of paramount importance. To highlight the strengths and limitations of the study by Liu et al ( 8), here, we review basic ML methodology with a focus on DL model implementation and recent guidelines for AI clinical trial design and reporting.

show abstract

Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Cited by 127 publications

References 18 publications

Digital auscultation as a diagnostic aid to detect childhood pneumonia: A systematic review

Digital auscultation as a diagnostic aid to detect childhood pneumonia: A systematic review

Quality use of artificial intelligence in medical imaging: What do radiologists need to know?

Clinical Machine Learning Modeling Studies: Methodology and Data Reporting

Contact Info

Product

Resources

About