PI Prob: A risk prediction and clinical guidance system for evaluating patients with recurrent infections

Abstract

Background Primary immunodeficiency diseases represent an expanding set of heterogeneous conditions which are difficult to recognize clinically. Diagnostic rates outside of the newborn period have not changed appreciably. This concern underscores a need for novel methods of disease detection. Objective We built a Bayesian network to provide real-time risk assessment about primary immunodeficiency and to facilitate prescriptive analytics for initiating the most appropriate diagnostic work up. Our goal is to improve diagnostic rates for primary immunodeficiency and shorten time to diagnosis. We aimed to use readily available health record data and a small training dataset to prove utility in diagnosing patients with relatively rare features. Methods We extracted data from the Texas Children’s Hospital electronic health record on a large population of primary immunodeficiency patients (n = 1762) and appropriately-matched set of controls (n = 1698). From the cohorts, clinically relevant prior probabilities were calculated enabling construction of a Bayesian network probabilistic model(PI Prob). Our model was constructed with clinical-immunology domain expertise, trained on a balanced cohort of 100 cases-controls and validated on an unseen balanced cohort of 150 cases-controls. Performance was measured by area under the receiver operator characteristic curve (AUROC). We also compared our network performance to classic machine learning model performance on the same dataset. Results PI Prob was accurate in classifying immunodeficiency patients from controls (AUROC = 0.945; p<0.0001) at a risk threshold of ≥6%. Additionally, the model was 89% accurate for categorizing validation cohort members into appropriate International Union of Immunological Societies diagnostic categories. Our network outperformed 3 other machine learning models and provides superior transparency with a prescriptive output element. Conclusion Artificial intelligence methods can classify risk for primary immunodeficiency and guide management. PI Prob enables accurate, objective decision making about risk and guides the user towards the appropriate diagnostic evaluation for patients with recurrent infections. Probabilistic models can be trained with small datasets underscoring their utility for rare disease detection given appropriate domain expertise for feature selection and network construction.

Who’s your data? Primary immune deficiency differential diagnosis prediction via machine learning and data mining of the USIDNET registry

“…In the past few years, we have toyed with different approaches to machine learning classi cation, including linear discriminant analysis, decision tree and random forests (9). In the eld of PID diseases, one group in Finland (10)(11)(12) and another in Houston at Baylor (13) have attempted machine learningassisted classi cation and prediction models, with promising results.…”

Large-scale deep learning analysis for the early diagnosis of primary immunodeficiencies

“…To the best of our knowledge, we were the rst to interrogate large EHR data for the early diagnosis of CID/CVID and the identi cation of antecedent phenotype combinations through deep learning and OR calculations. Moreover, none of the previous studies focused on CID [24][25][26][27] . The most recent study developed an LR model using 6,422 patients, of whom 247 were diagnosed with PID 24 .…”

“…with PIDs) 25 . Abyazi et al, identi ed different proteomic pro les in patients with noninfectious complications against uncomplicated CVID, implementing unsupervised learning in 72 participants 26 .…”

Natural language processing of clinical notes enables early inborn error of immunity risk ascertainment

“… 6 , 7 , 8 While education campaigns remain critical for raising awareness about IEI, leveraging health system data has shown promise for predicting immunologic risk across populations. 8 , 9 , 10 , 11 Machine learning (ML) and other computational approaches are emerging to enable earlier and more accurate diagnoses. 8 , 12 , 13 To date, application of ML and artificial intelligence has been limited to analyzing structured electronic health record (EHR), laboratory, and genomic data; however, use of unstructured data such as free text remains largely untapped as a resource.…”