Evaluation of machine learning pipeline for blood culture outcome prediction on prospectively collected data in Western Australian emergency department

Abstract

Bloodstream infections (BSIs) are particularly problematic in the emergency department (ED) of hospitals, where patients often present with undifferentiated illness, and the presence of BSI is hard to detect. Identifying whether a patient needs a blood culture (BC) test performed is one component of this challenge with implications for diagnostic efficiency, stewardship, and unnecessary resource expenditure. This paper explores the validation of previously developed machine learning (ML) models for BC outcome prediction using a prospectively collected ED patient dataset. An ML pipeline containing models previously trained using complete blood count (CBC), white blood cell differential (DIFF), and cell population data (CPD) generated by Sysmex XN-2000 haematology analysers was further evaluated using prospectively collected data containing patient sample results from the ED at Sir Charles Gairdner hospital (SCGH), Perth, Western Australia. Blood samples used to produce CBC,DIFF, and CPD were obtained at the same time as BC samples. There were 64 samples from 63 unique patients. 54 of those samples were associated with negative BC results, and 10 with positive BC results. We evaluated previously developed XGBoost (XG) and Random Forest (RF) ML models for positive BC outcome prediction. The RF and XG models obtained area under the receiver operating characteristic curve scores of 0.870 and 0.841 on the ED dataset. These results provide the foundation for further validation and shadow deployments of BC outcome prediction models in clinical settings. ML models,data, and code for reproducing the results presented in this work is provided in an open-source repository.