Refinement and Validation of a Clinical-Based Approach to Evaluate Young Febrile Infants

Abstract

BACKGROUND AND OBJECTIVE: For febrile infants, predictive models to detect bacterial infections are available, but clinical adoption remains limited by implementation barriers. There is a need for predictive models using widely available predictors. Thus, we previously derived 2 novel predictive models (machine learning and regression) by using demographic and clinical factors, plus urine studies. The objective of this study is to refine and externally validate the predictive models. METHODS: This is a cross-sectional study of infants initially evaluated at one pediatric emergency department from January 2011 to December 2018. Inclusion criteria were age 0 to 90 days, temperature ≥38°C, documented gestational age, and insurance type. To reduce potential biases, we derived models again by using derivation data without insurance status and tested the ability of the refined models to detect bacterial infections (ie, urinary tract infection, bacteremia, and meningitis) in the separate validation sample, calculating areas-under-the-receiver operating characteristic curve, sensitivities, and specificities. RESULTS: Of 1419 febrile infants (median age 53 days, interquartile range = 32–69), 99 (7%) had a bacterial infection. Areas-under-the-receiver operating characteristic curve of machine learning and regression models were 0.92 (95% confidence interval [CI] 0.89–0.94) and 0.90 (0.86–0.93) compared with 0.95 (0.91–0.98) and 0.96 (0.94–0.98) in the derivation study. Sensitivities and specificities of machine learning and regression models were 98.0% (94.7%–100%) and 54.2% (51.5%–56.9%) and 96.0% (91.5%–99.1%) and 50.0% (47.4%–52.7%). CONCLUSIONS: Compared with the derivation study, the machine learning and regression models performed similarly. Findings suggest a clinical-based model can estimate bacterial infection risk. Future studies should prospectively test the models and investigate strategies to optimize clinical adoption.

Derivation of a clinical‐based model to detect invasive bacterial infections in febrile infants

“…This is potentially concerning because the AAP CPG classifies ill‐appearing infants as high risk regardless of other characteristics 3 . As in our previous work, 5,6 we included ill‐appearing infants because there may be a subgroup of ill‐appearing Intermediate or low‐risk infants for whom management could be altered (e.g., hospitalizing, but holding antimicrobial therapy). Among all infants with IBIs, the misclassified infant is the only one who did not receive a urinalysis.…”

“…We sought to mitigate these potential sources of bias by implementing a rigorous and iterative data abstraction process with high inter‐rater reliability. Additionally, our prior work suggests that the frequency of documentation errors may be low and/or may be systematic, reducing the effects of this potential source of bias 5,6 . Additionally, despite the relatively large sample size, there are few IBI outcomes.…”

“…It is challenging to predict rare outcomes such as IBIs in febrile infants. 24 Based on our previous work, 5,6 infants with bacterial infections (i.e., UTIs, bacteremia, bacterial meningitis) share many characteristics with infants with IBIs. For these reasons, we used the broader category of bacterial infection as a surrogate outcome to build the models.…”

“…To create the sample, we combined study samples from previous studies 5,6 . As described previously, we used a standard protocol to manually review ED encounters to identify eligible infants and abstract predictor variables.…”

“…For chronic medical conditions, two authors reviewed records for each infant, categorizing conditions as present or absent based on the likelihood that the diagnosis would predispose an infant to IBI. The two authors discussed discrepant categorizations until they reached a consensus 5,6 . As done previously, we used an evidence‐based collection of keywords to distinguish ill from well appearance 5,6,19 .…”