Human–machine teaming is key to AI adoption: clinicians’ experiences with a deployed machine learning system

Henry, Katharine; Kornfield, Rachel; Sridharan, Anirudh; Linton, Robert C.; Groh, Catherine; Wang, Tony; Wu, Albert W.; Mutlu, Bilge; Saria, Suchi

doi:10.1038/s41746-022-00597-7

Cited by 94 publications

(101 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our companion paper, we used EHR data to examine various patient, provider and environmental factors associated with the adoption of TREWS 22 . In a further separate study, we analyzed qualitative impressions of TREWS and factors influencing its integration into workflow through semi-structured interviews with providers using the system 33 .…”

Section: Discussionmentioning

confidence: 99%

Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis

Adams

Henry

Sridharan

et al. 2022

Nat Med

Self Cite

126

102

View full text Add to dashboard Cite

Early recognition and treatment of sepsis are linked to improved patient outcomes. Machine learning-based early warning systems may reduce the time to recognition, but few systems have undergone clinical evaluation. In this prospective, multi-site cohort study, we examined the association between patient outcomes and provider interaction with a deployed sepsis alert system called the Targeted Real-time Early Warning System (TREWS). During the study, 590,736 patients were monitored by TREWS across five hospitals. We focused our analysis on 6,877 patients with sepsis who were identified by the alert before initiation of antibiotic therapy. Adjusting for patient presentation and severity, patients in this group whose alert was confirmed by a provider within 3 h of the alert had a reduced in-hospital mortality rate (3.3%, confidence interval (CI) 1.7, 5.1%, adjusted absolute reduction, and 18.7%, CI 9.4, 27.0%, adjusted relative reduction), organ failure and length of stay compared with patients whose alert was not confirmed by a provider within 3 h. Improvements in mortality rate (4.5%, CI 0.8, 8.3%, adjusted absolute reduction) and organ failure were larger among those patients who were additionally flagged as high risk. Our findings indicate that early warning systems have the potential to identify sepsis patients early and improve patient outcomes and that sepsis patients who would benefit the most from early treatment can be identified and prioritized at the time of the alert

show abstract

Section: Discussionmentioning

confidence: 99%

Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis

Adams

Henry

Sridharan

et al. 2022

Nat Med

Self Cite

126

102

View full text Add to dashboard Cite

show abstract

“…That is, one should be able to bucket patients into the proposed study and control arms from any random time point and arrive at similar differences in mortality. Furthermore, we hypothesize that by virtue of delaying alerts until verifiable symptoms were present 3 (i.e., ‘looking over the shoulder ‘ of clinicians 5 ), rather than helping to anticipate or predict cases of sepsis based on sparse or incomplete data, the TREWS alerts are increasingly correlated with the parallel and ongoing processes of care, which may alone explain the decisions to evaluate and treat patients. In such a setting, any causal claims of the potential effect of TREWS on patient outcomes may require adjustments by indicators of the processes of care (Hypothesis II); a classic backdoor adjustment problem in the causal inference literature 9 .…”

Section: Figurementioning

confidence: 99%

“…combined effect of early recognition and patient evaluation, referred to as human-machine teaming 3 . We have reproduced their cohort characteristics in Figure 1.…”

mentioning

confidence: 99%

“…While Adams et al do not explicitly state the mechanism by which the study intervention improves outcomes, they presumably believe one of the following to be responsible: 1) early recognition of sepsis provided by the TREWS alert followed by clinician acknowledgement and patient evaluation, 2) a streamlined user interface (UI) for guiding caregivers through a protocolized sepsis care bundle 4 , or 3) the combined effect of early recognition and patient evaluation, referred to as human-machine teaming 3 . We have reproduced their cohort characteristics in Figure 1.…”

mentioning

confidence: 99%

“…streamlined user interface (UI) for guiding caregivers through a protocolized sepsis care bundle 4 , or 3) the combined effect of early recognition and patient evaluation, referred to as human-machine teaming 3 . We have reproduced their cohort characteristics in Figure 1.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Randomized Clinical Trials or Convenient Controls: TREWS or FALSE?

Nemati

Shashikumar

Holder

et al. 2022

Preprint

View full text Add to dashboard Cite

We read with interest the Adams et al. (doi: https://doi.org/10.1038/s41591-022-01894-0) report of the TREWS machine learning (ML)-based sepsis early warning system. The authors conclude that large-scale randomized trials are needed to confirm their observations, but assert that their findings indicate the potential for the TREWS system to identify sepsis patients early and improve patient outcomes, including a significant decrease in mortality. However, this conclusion is based upon a comparison of those whose alert was confirmed vs. not confirmed within 3 hours, rather than random allocation to TREWS vs. no TREWS. Using data from over 650,000 patient encounters across two distinct healthcare systems, we show that the findings of Adams et al. are likely to be severely biased due to the failure to adjust for 'processes of care'-related confounding factors.

show abstract

A Nationwide Network of Health AI Assurance Laboratories

Shah,

Halamka,

Saria

et al. 2024

JAMA

Self Cite

View full text Add to dashboard Cite

ImportanceGiven the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed.ObservationsWhile there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings.Conclusion and RelevanceThe need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

show abstract

Human–machine teaming is key to AI adoption: clinicians’ experiences with a deployed machine learning system

Cited by 94 publications

References 42 publications

Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis

Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis

Randomized Clinical Trials or Convenient Controls: TREWS or FALSE?

A Nationwide Network of Health AI Assurance Laboratories

Contact Info

Product

Resources

About