Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal

Abstract

Background and purpose -Artificial intelligence (AI), deep learning (DL), and machine learning (ML) have become common research fields in orthopedics and medicine in general. Engineers perform much of the work. While they gear the results towards healthcare professionals, the difference in competencies and goals creates challenges for collaboration and knowledge exchange. We aim to provide clinicians with a context and understanding of AI research by facilitating communication between creators, researchers, clinicians, and readers of medical AI and ML research.Methods and results -We present the common tasks, considerations, and pitfalls (both methodological and ethical) that clinicians will encounter in AI research. We discuss the following topics: labeling, missing data, training, testing, and overfitting. Common performance and outcome measures for various AI and ML tasks are presented, including accuracy, precision, recall, F1 score, Dice score, the area under the curve, and ROC curves. We also discuss ethical considerations in terms of privacy, fairness, autonomy, safety, responsibility, and liability regarding data collecting or sharing.Interpretation -We have developed guidelines for reporting medical AI research to clinicians in the run-up to a broader consensus process. The proposed guidelines consist of a Clinical Artificial Intelligence Research (CAIR) checklist and specific performance metrics guidelines to present and evaluate research using AI components. Researchers, engineers, clinicians, and other stakeholders can use these proposal guidelines and the CAIR checklist to read, present, and evaluate AI research geared towards a healthcare setting.Machine learning (ML), deep learning (DL), and artificial intelligence (AI) have become increasingly common in orthopedics and other medical fields. Artificial intelligence, defined in 1955, is "the science and engineering of making intelligent machines," where intelligence is "the ability to learn and perform suitable techniques to solve problems and achieve goals, appropriate to the context in an uncertain, ever-varying world" (Manning 2020).Machine learning implies models and algorithms that learn from data rather than following explicit rules. Deep learning (DL) is a form of ML that uses large and multilayered artificial neural networks. Neural networks are computational algorithms influenced by biological networks for information processing. They consist of several layers of "neurons" that communicate. By training the neurons how to communicate, interactions develop that solve a particular problem. DL is currently the most successful and general ML approach (Michie et al. 1994, Manning 2020.Recent technological breakthroughs in computational hardware (like specialized graphics processors [GPUs] and cloud Key concepts presented in this review • Introduction to artificial intelligence (AI) and machine learning (ML) and how these relate to traditional clinical research statistics • Common pitfalls in AI research • How to measure and interpret AI and ML ...

Guidelines for Artificial Intelligence in Medicine: Literature Review and Content Analysis of Frameworks

“…All but three frameworks [19,34,35] identified greater than one intended audience, and typical audiences included AI developers, investigators, clinicians, patients, and policymakers. Frameworks provided either general guidance on the use of AI in medicine, typically in narrative prose (herein referred to as "descriptive frameworks") [19,32,[34][35][36][37] or guidance specifically on the reporting of AI studies in medicine, typically in checklist style (herein referred to as "reporting frameworks") [2,17,20,31,33,[38][39][40]. [36] Investigators, health care organizations Describes an evaluation framework for the application of AI in medicine Evaluating AI, Park et al [37] Clinicians Describes an approach for assessing published literature using AI for medical diagnoses Users' Guide, Liu et al [19] Health care organizations Describes barriers to the implementation of AI in medicine and provides solutions to address them Reporting and Implementing Interventions, Bates et al [35]…”

Narrative Review of Machine Learning in Rheumatic and Musculoskeletal Diseases for Clinicians and Researchers: Biases, Goals, and Future Directions

“…In recognition of the specific issues around prediction models utilizing AI and ML, extensions of these tools, TRIPOD-ML, TRIPOD-AI (reporting guideline) and PROBAST-AI (critical appraisal tool) are currently under development 60 . Other tools developed for fields such as cardiology 62 , orthopaedics 63 or for clinical trials 64,65 are also available.…”

Predictive models for clinical decision making: Deep dives in practical machine learning

“…We opted to exclude articles organised as checklists for reporting [8, 9], or tools for reading and evaluating the scientific quality of machine‐learning applications [10–12]. Nevertheless, some of these articles may be of interest to clinical researchers.…”

“…The main tutorial elements in the selected papers included a guided approach for designing a machine learning project [5] and a hands-on introduction to implementing machine learning algorithms in R statistical software [7]. We opted to exclude articles organised as checklists for reporting [8,9], or tools for reading and evaluating the scientific quality of machinelearning applications [10][11][12]. Nevertheless, some of these articles may be of interest to clinical researchers.…”