Diagnostic stewardship for blood cultures in the emergency department: A multicenter validation and prospective evaluation of a machine learning prediction tool

Schinkel, Michiel; Boerman, Anneroos W.; Bennis, Frank C.; Minderhoud, Tanca C.; Lie, Mei; Peters‐Sengers, Hessel; Holleman, F.; Schade, Rogier P.; Jonge, Robert de; Wiersinga, W. Joost; Nanayakkara, Prabath W.B.

doi:10.1016/j.ebiom.2022.104176

Cited by 21 publications

(22 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The selection process, including reasons for exclusion at each stage, is outlined in the PRISMA flow diagram (Figure 1). Notably, the majority (29 out of 30) of the included articles employed retrospective study designs, with only two conducting prospective validations of their models [40, 41]. A single study presented a prospective cohort design [29].…”

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

“…Most of the reviewed studies were conducted within a single institution; however, three studies utilized datasets encompassing two hospital systems [20, 22, 44], and two studies expanded their analysis to incorporate multi-center data [41, 49]. External validation, which is critical for the generalizability of findings, was performed in four studies [38, 40, 41, 44]. Compliance with the Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines, which enhance the reliability of predictive modeling, was confirmed in five studies [20, 28, 37, 39, 41, 50].…”

Section: Resultsmentioning

confidence: 99%

“…External validation, which is critical for the generalizability of findings, was performed in four studies [38, 40, 41, 44]. Compliance with the Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines, which enhance the reliability of predictive modeling, was confirmed in five studies [20, 28, 37, 39, 41, 50]. Data diversity is key in model training and validation.…”

Section: Resultsmentioning

confidence: 99%

See 3 more Smart Citations

Recent Advancements in Machine Learning-Based Bloodstream Infection Prediction: A Systematic Review and Meta-analysis of Diagnostic Test Accuracy

Bopche,

Damås

2024

Preprint

View full text Add to dashboard Cite

Abstract. Purpose: Bloodstream infections (BSIs) present significant public health challenges. With the advent of machine learning (ML), promising predictive models have been developed. This study evaluates their performance through a systematic review and meta-analysis. Methods: We performed a comprehensive systematic review across multiple databases, including PubMed, IEEE Xplore, ScienceDirect, ACM Digital Library, SpringerLink, Web of Science, Scopus, and Google Scholar. Eligible studies focused on BSIs within any hospital setting, employing ML models as the diagnostic test. We evaluated the risk of bias with the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist and assessed the quality of evidence using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach. Models reporting the area under the receiver operating characteristic curve (AUROC) were included in the meta-analysis to identify key performance drivers. Results: After screening, a total of 30 studies were eligible for synthesis, from which 41 models and 8 data types were extracted. Most of the studies were carried out in the inpatient settings (n=17; 56%), followed by the emergency department (ED) settings (n=7; 23%), and followed by the ICU settings (n=6; 20%). The reported AUROCs in the hospital inpatients settings, ranged from 0.51-0.866, in the ICU settings AUROCs ranged from 0.668-0.970, and in the emergency department (ED) settings the AUROCs of the models ranged from 0.728-0.844. One study reported prospective cohort study, while two prospectively validated their models. In the meta-analysis, laboratory tests, Complete Blood Count/Differential Count (CBC/DC), and ML model type contributed the most to model performance. Conclusion: This systematic review and meta-analysis show that on retrospective data, individual ML models can accurately predict BSIs at different stages of patient trajectory. Although they enable early prediction of BSI, a comprehensive approach to integrate data types and models is necessary. Systematic reporting, externally validated, and clinical implementation studies are needed to establish clinical confidence.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Recent Advancements in Machine Learning-Based Bloodstream Infection Prediction: A Systematic Review and Meta-analysis of Diagnostic Test Accuracy

Bopche,

Damås

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Editorials and reviews have attempted to define the term [9,[15][16][17], and some have detailed the various stages where interventions may be applied in the pre-analytical, analytical and post-analytical phases [9]. However, publications, including systematic reviews [18], retrospective studies [19,20] and multi-centre validation of machine learning tools [21], have used the term without defining it. One collaborative study regarding paediatric blood cultures (BCs) defined diagnostic stewardship as "optimising the use of diagnostic tests to improve treatment decisions" [22].…”

Section: Introductionmentioning

confidence: 99%

Diagnostic stewardship in infectious diseases: a scoping review

Shorten,

Pickering,

Goolden

et al. 2024

Journal of Medical Microbiology

View full text Add to dashboard Cite

Introduction. The term ‘diagnostic stewardship’ is relatively new, with a recent surge in its use within the literature. Despite its increasing popularity, a precise definition remains elusive. Various attempts have been made to define it, with some viewing it as an integral part of antimicrobial stewardship. The World Health Organization offers a broad definition, emphasizing the importance of timely, accurate diagnostics. However, inconsistencies in the use of this term still persist, necessitating further clarification. Gap Statement. There are currently inconsistencies in the definition of diagnostic stewardship used within the academic literature. Aim. This scoping review aims to categorize the use of diagnostic stewardship approaches and define this approach by identifying common characteristics and factors of its use within the literature. Methodology. This scoping review undertook a multi-database search from date of inception until October 2022. Any observational or experimental study where the authors define the intervention to be diagnostic stewardship from any clinical area was included. Screening of all papers was undertaken by a single reviewer with 10% verification by a second reviewer. Data extraction was undertaken by a single reviewer using a pre-piloted form. Given the wide variation in study design and intervention outcomes, a narrative synthesis approach was applied. Studies were clustered around common diagnostic stewardship interventions where appropriate. Results. After duplicate removal, a total of 1310 citations were identified, of which, after full-paper screening, 105 studies were included in this scoping review. The classification of an intervention as taking a diagnostic stewardship approach is a relatively recent development, with the first publication in this field dating back to 2017. The majority of research in this area has been conducted within the USA, with very few studies undertaken outside this region. Visual inspection of the citation map reveals that the current evidence base is interconnected, with frequent references to each other’s work. The interventions commonly adopt a restrictive approach, utilizing hard and soft stops within the pre-analytical phase to restrict access to testing. Upon closer examination of the outcomes, it becomes evident that there is a predominant focus on reducing the number of tests rather than enhancing the current test protocol. This is further reflected in the limited number of studies that report on test performance (including protocol improvements, specificity and sensitivity). Conclusion. Diagnostic stewardship seems to have deviated from its intended course, morphing into a rather rudimentary instrument wielded not to enhance but to constrict the scope of testing. Despite the World Health Organization’s advocacy for an ideology that promotes a more comprehensive approach to quality improvement, it may be more appropriate to consider alternative regional narratives when categorizing these types of quality improvement interventions.

show abstract

Blood culture algorithm implementation in emergency department patients as a diagnostic stewardship intervention

Theophanous,

Ramos,

Calland

et al. 2024

American Journal of Infection Control

View full text Add to dashboard Cite

Diagnostic stewardship for blood cultures in the emergency department: A multicenter validation and prospective evaluation of a machine learning prediction tool

Cited by 21 publications

References 30 publications

Recent Advancements in Machine Learning-Based Bloodstream Infection Prediction: A Systematic Review and Meta-analysis of Diagnostic Test Accuracy

Recent Advancements in Machine Learning-Based Bloodstream Infection Prediction: A Systematic Review and Meta-analysis of Diagnostic Test Accuracy

Diagnostic stewardship in infectious diseases: a scoping review

Blood culture algorithm implementation in emergency department patients as a diagnostic stewardship intervention

Contact Info

Product

Resources

About