Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework

Abstract

Objective To derive a comprehensive implementation framework for clinical AI models within hospitals informed by existing AI frameworks and integrated with reporting standards for clinical AI research. Materials and Methods (1) Derive a provisional implementation framework based on the taxonomy of Stead et al and integrated with current reporting standards for AI research: TRIPOD, DECIDE-AI, CONSORT-AI. (2) Undertake a scoping review of published clinical AI implementation frameworks and identify key themes and stages. (3) Perform a gap analysis and refine the framework by incorporating missing items. Results The provisional AI implementation framework, called SALIENT, was mapped to 5 stages common to both the taxonomy and the reporting standards. A scoping review retrieved 20 studies and 247 themes, stages, and subelements were identified. A gap analysis identified 5 new cross-stage themes and 16 new tasks. The final framework comprised 5 stages, 7 elements, and 4 components, including the AI system, data pipeline, human-computer interface, and clinical workflow. Discussion This pragmatic framework resolves gaps in existing stage- and theme-based clinical AI implementation guidance by comprehensively addressing the what (components), when (stages), and how (tasks) of AI implementation, as well as the who (organization) and why (policy domains). By integrating research reporting standards into SALIENT, the framework is grounded in rigorous evaluation methodologies. The framework requires validation as being applicable to real-world studies of deployed AI models. Conclusions A novel end-to-end framework has been developed for implementing AI within hospital clinical practice that builds on previous AI implementation frameworks and research reporting standards.

AImedReport: A Prototype Tool to Facilitate Research Reporting and Translation of AI Technologies in Healthcare

“…We suggest that AImedReport could further contribute to such implementation endeavors as a valuable resource. Planned future work will continue to converge with and align to various frameworks, like the SALIENT framework, 18 ABCDS, 22 and organizations, including the Office of the National Coordinator, 23 Food & Drug Administration, 24 Coalition for Health AI, 25 National Academy of Medicine, 5 Health AI Partnership, 26 National Institute of Standards and Technology, 27 World Health Organization, 28 and others.…”

“…This alignment was established to support compliance with reporting standards and provide a reference for the entire AI development lifecycle, aiding in informing development phases, engaging stakeholders, and supporting interpretability, knowledge continuity, transparency, and trust. While based on the Overgaard et al 1 framework, AImedReport’s matured versatility allows it to potentially suit other frameworks such as van der Vegt’s 18 SALIENT framework for broader AI implementation.…”

Enhancing Medical Decision Making: A Semantic Technology-Based Framework for Efficient Diagnosis Inference

“…In the current follow-up study, similar to other researchers [15], we evaluated our framework and examined its applicability and effectiveness via several defined measures, after we fully implemented it. As other researchers clarified [43,44], there is a challenge in the implementation, assessment, and integration of medical frameworks utilizing AI tools. As our framework shares a common objective with other systems (i.e., focusing on assisting and providing recommendations for patient diagnosis), a comparison is required (see Section 3).…”

Why clinical artificial intelligence is (almost) non‐existent in Australian hospitals and how to fix it

“…To explore this question further, we conducted a scoping review of clinical AI implementation guidelines, standards and frameworks and identified 20 published articles since 2019 from seven countries. 16 We found there were common stages to AI implementation to ensure the safe, effective and equitable introduction of AI into clinical practice. Although these stages vary, they generally always include a stage for problem definition to check that AI is needed and possible (stage I); retrospective (in silico or laboratory) evaluation to ensure that AI meets minimum performance requirements (stage II); prospective evaluation, often called a silent trial using real-time EMR data, to evaluate in an environment with zero patient risk the clinical utility, live AI performance, alert functionality, user interface design and the data quality and latency impact (stage III); a pilot trial to assess patient safety and clinical workflow integration (stage IV); and a larger clinical trial or rollout (stage V).…”

“…We integrated these standards into each stage of implementation and, together with broader findings from 20 other international frameworks, derived an end-to-end clinical AI implementation framework called SALIENT (Box 1). 16 SALIENT is the only framework with full coverage of all reporting guidelines so that it may provide a starting place for establishing that AI is tested and suitable for implementing into Australian health care.…”