A survey of extant organizational and computational setups for deploying predictive models in health systems

Kashyap, Sehj; Morse, Keith; Patel, Birju; Shah, Nigam H.

doi:10.1093/jamia/ocab154

Cited by 17 publications

(14 citation statements)

References 27 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reviewed body of literature was fairly recent, with the majority of the studies (32/45, 71%) having been published between 2020 and 2022 [ 45 - 76 ]. Most (33/45, 73%) of the articles were from North America and Europe [ 46 , 47 , 49 - 55 , 57 , 58 , 61 - 63 , 67 - 70 , 73 - 87 ], of which most (18/33, 55%) were from the United States [ 46 , 47 , 49 - 52 , 54 , 68 , 73 - 77 , 79 - 81 , 84 , 87 ]. The greatest number of AI systems were implemented either in hospital-wide settings (6/45, 13%) [ 50 , 55 , 56 , 65 , 74 , 80 ] or in radiology (6/45, 13%) [ 53 , 56 , 66 , 68 , 73 , 76 ].…”

Section: Resultsmentioning

confidence: 99%

“…The greatest number of AI systems were implemented either in hospital-wide settings (6/45, 13%) [ 50 , 55 , 56 , 65 , 74 , 80 ] or in radiology (6/45, 13%) [ 53 , 56 , 66 , 68 , 73 , 76 ]. Most (27/45, 60%) of the studies were authored by a multidisciplinary team [ 46 , 47 , 50 - 55 , 58 , 59 , 61 , 62 , 64 , 67 , 69 , 70 , 72 , 74 , 75 , 78 - 80 , 82 , 86 - 89 ], with clinical and IT or informatics backgrounds being the most common combination (9/27, 33%) [ 47 , 50 , 55 , 61 , 70 , 74 , 79 , 87 , 89 ]. Among studies with authors from only 1 domain, the most common background was clinical (8/45, 18%) [ 63 , 65 , 66 , 68 , 71 , 73 , 76 , 84 ].…”

Section: Resultsmentioning

confidence: 99%

“…. Most (33/45, 73%) of the articles were from North America and Europe[46,47,[49][50][51][52][53][54][55]57,58,[61][62][63][67][68][69][70][73][74][75][76][77][78][79][80][81][82][83][84][85][86][87], of which most (18/33, 55%) were from the United States[46,47,49-52,54,68,73-77, 79-81,84,87]. The greatest number of AI systems were implemented either in hospital-wide settings (6/45, 13%)[50,55,56,65,74,80] or in radiology (6/45, 13%)[53,56,66,68,73,76].…”

mentioning

confidence: 99%

“…59,61,62,64,67,69,70,72,74,75,[78][79][80]82,[86][87][88][89], with clinical and IT or informatics backgrounds being the most common combination (9/27, 33%)[47,50,55,61,70,74,79,87,89]. Among studies with authors from only 1 domain, the most common background was clinical (8/45, 18%)[63,65,66,68,71,73,76,84].…”

mentioning

confidence: 99%

See 3 more Smart Citations

Artificial Intelligence Applications in Health Care Practice: Scoping Review

Sharma¹,

Savage²,

Nair³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

Background Artificial intelligence (AI) is often heralded as a potential disruptor that will transform the practice of medicine. The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI and an exponential growth of publications. However, the development of AI applications does not guarantee their adoption into routine practice. There is a risk that despite the resources invested, benefits for patients, staff, and society will not be realized if AI implementation is not better understood. Objective The aim of this study was to explore how the implementation of AI in health care practice has been described and researched in the literature by answering 3 questions: What are the characteristics of research on implementation of AI in practice? What types and applications of AI systems are described? What characteristics of the implementation process for AI systems are discernible? Methods A scoping review was conducted of MEDLINE (PubMed), Scopus, Web of Science, CINAHL, and PsycINFO databases to identify empirical studies of AI implementation in health care since 2011, in addition to snowball sampling of selected reference lists. Using Rayyan software, we screened titles and abstracts and selected full-text articles. Data from the included articles were charted and summarized. Results Of the 9218 records retrieved, 45 (0.49%) articles were included. The articles cover diverse clinical settings and disciplines; most (32/45, 71%) were published recently, were from high-income countries (33/45, 73%), and were intended for care providers (25/45, 56%). AI systems are predominantly intended for clinical care, particularly clinical care pertaining to patient-provider encounters. More than half (24/45, 53%) possess no action autonomy but rather support human decision-making. The focus of most research was on establishing the effectiveness of interventions (16/45, 35%) or related to technical and computational aspects of AI systems (11/45, 24%). Focus on the specifics of implementation processes does not yet seem to be a priority in research, and the use of frameworks to guide implementation is rare. Conclusions Our current empirical knowledge derives from implementations of AI systems with low action autonomy and approaches common to implementations of other types of information systems. To develop a specific and empirically based implementation framework, further research is needed on the more disruptive types of AI systems being implemented in routine care and on aspects unique to AI implementation in health care, such as building trust, addressing transparency issues, developing explainable and interpretable solutions, and addressing ethical concerns around privacy and data protection.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Artificial Intelligence Applications in Health Care Practice: Scoping Review

Sharma¹,

Savage²,

Nair³

et al. 2022

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…Such audits can be performed by internal organizational teams responsible for deploying predictive models in healthcare (23, 55), with the caveat that internal audits may have limited independence and objectivity (23). Alternatively, regulators may conduct such audits, such as the Food and Drug Administration (FDA)’s proposed Digital Health Software Precertification Program which evaluates real world performance of software as a medical device (56).…”

Section: Discussionmentioning

confidence: 99%

Considerations in the Reliability and Fairness Audits of Predictive Models for Advance Care Planning

Sattler

Wang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Multiple reporting guidelines for artificial intelligence (AI) models in healthcare recommend that models be audited for reliability and fairness. However, there is a gap of operational guidance for performing reliability and fairness audits in practice. Following guideline recommendations, we conducted a reliability audit of two models based on model performance and calibration as well as a fairness audit based on summary statistics, subgroup performance and subgroup calibration. We assessed the Epic End-of-Life (EOL) Index model and an internally developed Stanford Hospital Medicine (HM) Advance Care Planning (ACP) model in 3 practice settings: Primary Care, Inpatient Oncology and Hospital Medicine, using clinicians' answers to the surprise question as a surrogate outcome. For performance, the models had positive predictive value (PPV) at or above 0.76 in all settings. In Hospital Medicine and Inpatient Oncology, the Stanford HM ACP model had higher sensitivity (0.69, 0.89 respectively) than the EOL model (0.20, 0.27), and better calibration (O/E 1.5, 1.7) than the EOL model (O/E 2.5, 3.0). The Epic EOL model flagged fewer patients (11%, 21% respectively) than the Stanford HM ACP model (38%, 75%). There were no differences in performance and calibration by sex. Both models had lower sensitivity in Hispanic/Latino male patients with Race listed as "Other." In a follow-up survey, 10/10 clinicians reported that the summary statistics, overall performance, and subgroup performance would affect their decision to use the model to guide care, and 9/10 said the same for overall and subgroup calibration. The most commonly identified barriers for routinely conducting such reliability and fairness audits were poor demographic data quality and lack of data access. This audit required 115 person-hours across 8-10 months. Our recommendations for performing reliability and fairness audits before model deployment include verifying data validity, analyzing model performance on intersectional subgroups, and collecting clinician-patient linkages as necessary for manual chart review and label generation. Those responsible for AI models should require such audits before model deployment and mediate between model auditors and impacted stakeholders.

show abstract