Systematic comparison of published host gene expression signatures for bacterial/viral discrimination

Abstract

Background Measuring host gene expression is a promising diagnostic strategy to discriminate bacterial and viral infections. Multiple signatures of varying size, complexity, and target populations have been described. However, there is little information to indicate how the performance of various published signatures compare to one another. Methods This systematic comparison of host gene expression signatures evaluated the performance of 28 signatures, validating them in 4589 subjects from 51 publicly available datasets. Thirteen COVID-specific datasets with 1416 subjects were included in a separate analysis. Individual signature performance was evaluated using the area under the receiving operating characteristic curve (AUC) value. Overall signature performance was evaluated using median AUCs and accuracies. Results Signature performance varied widely, with median AUCs ranging from 0.55 to 0.96 for bacterial classification and 0.69–0.97 for viral classification. Signature size varied (1–398 genes), with smaller signatures generally performing more poorly (P < 0.04). Viral infection was easier to diagnose than bacterial infection (84% vs. 79% overall accuracy, respectively; P < .001). Host gene expression classifiers performed more poorly in some pediatric populations (3 months–1 year and 2–11 years) compared to the adult population for both bacterial infection (73% and 70% vs. 82%, respectively; P < .001) and viral infection (80% and 79% vs. 88%, respectively; P < .001). We did not observe classification differences based on illness severity as defined by ICU admission for bacterial or viral infections. The median AUC across all signatures for COVID-19 classification was 0.80 compared to 0.83 for viral classification in the same datasets. Conclusions In this systematic comparison of 28 host gene expression signatures, we observed differences based on a signature’s size and characteristics of the validation population, including age and infection type. However, populations used for signature discovery did not impact performance, underscoring the redundancy among many of these signatures. Furthermore, differential performance in specific populations may only be observable through this type of large-scale validation.

Prospective Validation of a Rapid Host Gene Expression Test to Discriminate Bacterial From Viral Respiratory Infection

“…In particular, test performance decreased as the number of clinical categories increased. 18 , 31 Therefore, the present study focused on the narrower indication of acute (≤7 days) febrile respiratory illness suspected to be caused by bacterial or viral infection.…”

Benchmarking transcriptional host response signatures for infection diagnosis

“…The scope of our study was fundamentally different from the scope of previous studies that focused on deriving potential signatures of viral or bacterial infections. Going beyond initial efforts to compare the robustness of existing signatures, 10 , 18 , 19 our evaluation framework is the first to provide a reference space where any arbitrary signature of infection can be rigorously assessed along two equally critical axes, robustness, and cross-reactivity. The framework is based on an extensive data curation of 17,105 blood transcriptional profiles from infectious and non-infectious conditions combined with a universal, model-free signature scoring method.…”

“…Such an evaluation is a complex task because it requires integrating and analyzing massive amounts of transcriptional studies involving the pathogen of interest along with a wide variety of other infectious and non-infectious conditions that may cause cross-reactivity. Despite recent progress in this direction, 10 , 18 , 19 a general framework to benchmark both robustness and cross-reactivity of candidate signatures is still lacking.…”

Antibiotic stewardship in the era of precision medicine

“… 9 For example, precision medicine approaches can help distinguish viral from bacterial pathogens in pneumonia, thus limiting the unnecessary use of antibacterial agents, a key goal in antibiotic stewardship. 10 Figure 1 presents an overview of the key components of antibiotic stewardship using a precision medicine approach. The technology with precision medicine is more rapid than traditional laboratory methods such as cultures, which may reduce the time to diagnosis and lead to more appropriate antibiotic therapy.…”