Governance of Clinical AI applications to facilitate safe and equitable deployment in a large health system: Key elements and early successes

Liao, Frank; Adelaine, Sabrina; Afshar, Majid; Patterson, Brian W.

doi:10.3389/fdgth.2022.931439

Cited by 17 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this reason, healthcare providers in the United States cannot assume that CDS systems have been validated , and healthcare systems should establish oversight procedures to independently ensure that these tools satisfy basic requirements of safety and quality. Some academic healthcare systems have started this effort, including Duke Medical Center ( https://aihealth.duke.edu/algorithm-based-clinical-decision-support-abcds/ , last accessed June 21, 2023), University of Wisconsin Health, 57 and Stanford Medicine. 58 These programs can serve as an example for others, though it is important to acknowledge that each provider is different: there is no “one-size-fits-all” approach to algorithmic governance.…”

Section: Discussionmentioning

confidence: 99%

A call for better validation of opioid overdose risk algorithms

McElfresh

Chen

Oliva

et al. 2023

Journal of the American Medical Informatics Association

View full text Add to dashboard Cite

Clinical decision support (CDS) systems powered by predictive models have the potential to improve the accuracy and efficiency of clinical decision-making. However, without sufficient validation, these systems have the potential to mislead clinicians and harm patients. This is especially true for CDS systems used by opioid prescribers and dispensers, where a flawed prediction can directly harm patients. To prevent these harms, regulators and researchers have proposed guidance for validating predictive models and CDS systems. However, this guidance is not universally followed and is not required by law. We call on CDS developers, deployers, and users to hold these systems to higher standards of clinical and technical validation. We provide a case study on two CDS systems deployed on a national scale in the United States for predicting a patient’s risk of adverse opioid-related events: the Stratification Tool for Opioid Risk Mitigation (STORM), used by the Veterans Health Administration, and NarxCare, a commercial system.

show abstract

Section: Discussionmentioning

confidence: 99%

A call for better validation of opioid overdose risk algorithms

McElfresh

Chen

Oliva

et al. 2023

Journal of the American Medical Informatics Association

View full text Add to dashboard Cite

show abstract

“…[15] This could be achieved through incentives for technology adoption, support for implementation research and technical development, and the development of evidence-based guidelines to ensure ethical and secure use of AI in healthcare. [60] However, the collaboration between HCP and AI is key to success in improving the accuracy, consistency, and completeness of medical documentation while minimizing documentation errors. [51,61] It is also important to develop operationalization and implementation plans with accountable, fair, and inclusive AI approaches to ensure the trustworthiness of the digital scribes.…”

Section: Discussionmentioning

confidence: 99%

Development and Evaluation of a Digital Scribe: Conversation Summarization Pipeline for Emergency Department Counseling Sessions towards Reducing Documentation Burden

Sezgin,

Sirrianni,

Kranz

2023

Preprint

View full text Add to dashboard Cite

ObjectiveWe present a proof-of-concept digital scribe system as an ED clinical conversation summarization pipeline and report its performance.Materials and MethodsWe use four pre-trained large language models to establish the digital scribe system: T5-small, T5-base, PEGASUS-PubMed, and BART-Large-CNN via zero-shot and fine-tuning approaches. Our dataset includes 100 referral conversations among ED clinicians and medical records. We report the ROUGE-1, ROUGE-2, and ROUGE-L to compare model performance. In addition, we annotated transcriptions to assess the quality of generated summaries.ResultsThe fine-tuned BART-Large-CNN model demonstrates greater performance in summarization tasks with the highest ROUGE scores (F1ROUGE-1=0.49, F1ROUGE-2=0.23, F1ROUGE-L=0.35) scores. In contrast, PEGASUS-PubMed lags notably (F1ROUGE-1=0.28, F1ROUGE-2=0.11, F1ROUGE-L=0.22). BART-Large-CNN’s performance decreases by more than 50% with the zero-shot approach. Annotations show that BART-Large-CNN performs 71.4% recall in identifying key information and a 67.7% accuracy rate.DiscussionThe BART-Large-CNN model demonstrates a high level of understanding of clinical dialogue structure, indicated by its performance with and without fine-tuning. Despite some instances of high recall, there is variability in the model’s performance, particularly in achieving consistent correctness, suggesting room for refinement. The model’s recall ability varies across different information categories.ConclusionThe study provides evidence towards the potential of AI-assisted tools in reducing clinical documentation burden. Future work is suggested on expanding the research scope with larger language models, and comparative analysis to measure documentation efforts and time.

show abstract

“…Our health system is an early adopter of AI governance with a review process similar to other health systems. 29,30 The Clinical AI and Predictive Analytics committee follows the minimum information about clinical artificial intelligence modeling (MI-CLAIM Checklist). 31 The offline validation of our model incorporated principles from multiple reporting guidelines on prediction models, bias and fairness, and validation.…”

Section: Discussionmentioning

confidence: 99%

Protocol for a Real-Time Electronic Health Record Implementation of a Natural Language Processing and Deep Learning Clinical Decision Support Tool: A Use-Case for an Opioid Misuse Screener in Hospitalized Adults

Afshar

Adelaine

Resnik

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The clinical narrative in the electronic health record (EHR) carries valuable information for predictive analytics, but its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing open-source NLP engines to provide interoperable and standardized CDS at the bedside. This clinical protocol describes a reproducible workflow for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. We apply the NLP CDS infrastructure to a use-case for hospital-wide opioid misuse screening using an open-source deep learning model that leverages clinical notes mapped to standardized medical vocabularies. The resultant NLP and deep learning pipeline can process clinical notes and provide decision support to the bedside within minutes of a provider entering a note into the EHR for all hospitalized patients. The protocol includes a human-centered design and an implementation framework with a cost-effectiveness and patient outcomes analysis plan.

show abstract

Governance of Clinical AI applications to facilitate safe and equitable deployment in a large health system: Key elements and early successes

Cited by 17 publications

References 13 publications

A call for better validation of opioid overdose risk algorithms

A call for better validation of opioid overdose risk algorithms

Development and Evaluation of a Digital Scribe: Conversation Summarization Pipeline for Emergency Department Counseling Sessions towards Reducing Documentation Burden

Protocol for a Real-Time Electronic Health Record Implementation of a Natural Language Processing and Deep Learning Clinical Decision Support Tool: A Use-Case for an Opioid Misuse Screener in Hospitalized Adults

Contact Info

Product

Resources

About