A multicenter evaluation of computable phenotyping approaches for SARS-CoV-2 infection and COVID-19 hospitalizations

Khera, Rohan; Mortazavi, Bobak J.; Sangha, Veer; Warner, Frederick; Young, H. Patrick; Ross, Joseph S.; Shah, Nilay D.; Theel, Elitza S.; Jenkinson, W. Garrett; Knepper, Camille; Wang, Karen; Peaper, David R.; Martinello, Richard A.; Brandt, Cynthia; Lin, Zhenqiu; Ko, Albert I.; Krumholz, Harlan M.; Pollock, Benjamin D.; Schulz, Wade L.

doi:10.1038/s41746-022-00570-4

Cited by 10 publications

(4 citation statements)

References 35 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 13 For defining a phenotype, one study found that standard tools like positive lab tests and ICD10 codes had limited precision and recall. 14 Our work is complementary to that work in showing the value of NLP to improve sensitivity and find additional cases. Other computable phenotyping work is in the context of post-acute sequelae SARS-CoV-2 infection (PASC), 15 and therefore must address the heterogeneity and uncertainty of the condition.…”

Section: Discussionmentioning

confidence: 69%

A computable case definition for patients with SARS-CoV2 testing that occurred outside the hospital

et al. 2023

View full text Add to dashboard Cite

Objective To identify a cohort of COVID-19 cases, including when evidence of virus positivity was only mentioned in the clinical text, not in structured laboratory data in the electronic health record (EHR). Materials and Methods Statistical classifiers were trained on feature representations derived from unstructured text in patient electronic health records (EHRs). We used a proxy dataset of patients with COVID-19 polymerase chain reaction (PCR) tests for training. We selected a model based on performance on our proxy dataset and applied it to instances without COVID-19 PCR tests. A physician reviewed a sample of these instances to validate the classifier. Results On the test split of the proxy dataset, our best classifier obtained 0.56 F1, 0.6 precision, and 0.52 recall scores for SARS-CoV2 positive cases. In an expert validation, the classifier correctly identified 97.6% (81/84) as COVID-19 positive and 97.8% (91/93) as not SARS-CoV2 positive. The classifier labeled an additional 960 cases as not havingt SARS-CoV2 lab tests in hospital, and only 177 of those cases had the ICD-10 code for COVID-19. Discussion Proxy dataset performance may be worse because these instances sometimes include discussion of pending lab tests. The most predictive features are meaningful and interpretable. The type of external test that was performed is rarely mentioned. Conclusion COVID-19 cases that had testing done outside of the hospital can be reliably detected from the text in EHRs. Training on a proxy dataset was a suitable method for developing a highly performant classifier without labor intensive labeling efforts. Lay Summary For a significant period at the start of the COVID-19 pandemic, some hospitals routinely tested every patient who came to the emergency room or was admitted for COVID-19 with a polymerase chain reaction (PCR) test. However, they may have skipped this test if the patient reported a recent positive test outside the hospital, and these patients would be treated as if they had tested positive at the hospital. These patients are hard to detect for later study, because hospitals will not have an electronic record of a positive test. In this work, we hypothesized that we could detect these patients by teaching machine learning methods to read the text in electronic health records, where positive tests would be mentioned by clinicians. We found that we could detect these patients with high accuracy, as validated by a clinician, that there are many additional cases that we can find this way, and that many of these cases would be hard to detect with other methods.

show abstract

Section: Discussionmentioning

confidence: 69%

A computable case definition for patients with SARS-CoV2 testing that occurred outside the hospital

et al. 2023

View full text Add to dashboard Cite

show abstract

“…This finding contrasts with early assessments of COVID-19 ICD-10 codes which reported excellent positive and negative predictive values for ICD-10 codes compared to PCR data in all patients and critically ill patients, respectively. 8,24,[31][32][33][34] In retrospect, the excellent performance for ICD-10 codes in these studies was likely due to the use of PCR positivity as the gold standard for COVID-19 hospitalization, as well as the newness of the epidemic, focal use of testing, low healthcare utilization for non-COVID-19 care (hence fewer incidental cases), and fewer false-positive results due to prior infections. We advise caution when interpreting studies which identify COVID-19 hospitalizations using ICD-10 codes during the current era.…”

Section: Discussionmentioning

confidence: 90%

“…[3][4][5] Large cohort studies have also used different approaches for defining COVID-19 hospitalizations, including a positive PCR alone, [6][7][8][9] International Classification of Disease, Tenth Revision, Clinical Modification (ICD-10-CM) codes for COVID-19, [10][11][12][13][14][15][16][17][18] institutional definitions, or combinations of these. [19][20][21][22][23][24][25] Notwithstanding the panoply of definitions being used, few data are available that compare estimates of COVID-19 hospitalizations, severity of illness, mortality, and trends between definitions, nor their accuracy in identifying primary or contributing versus incidental infections.…”

mentioning

confidence: 99%

Impact of changing case definitions for coronavirus disease 2019 (COVID-19) hospitalization on pandemic metrics

Shappell

Chan

Chen

et al. 2023

Infect. Control Hosp. Epidemiol.

View full text Add to dashboard Cite

Objective: To examine the impact of commonly used case definitions for coronavirus disease 2019 (COVID-19) hospitalizations on case counts and outcomes. Design, patients, and setting: Retrospective analysis of all adults hospitalized between March 1, 2020, and March 1, 2022, at 5 Massachusetts acute-care hospitals. Interventions: We applied 6 commonly used definitions of COVID-19 hospitalization: positive severe acute respiratory coronavirus virus 2 (SARS-CoV-2) polymerase chain reaction (PCR) assay within 14 days of admission, PCR plus dexamethasone administration, PCR plus remdesivir, PCR plus hypoxemia, institutional COVID-19 flag, or COVID-19 International Classification of Disease, Tenth Revision (ICD-10) codes. Outcomes included case counts and in-hospital mortality. Overall, 100 PCR-positive cases were reviewed to determine each definition’s accuracy for distinguishing primary or contributing versus incidental COVID-19 hospitalizations. Results: Of 306,387 hospital encounters, 15,436 (5.0%) met the PCR-based definition. COVID-19 hospitalization counts varied substantially between definitions: 4,628 (1.5% of all encounters) for PCR plus dexamethasone, 5,757 (1.9%) for PCR plus remdesivir, 11,801 (3.9%) for PCR plus hypoxemia, 15,673 (5.1%) for institutional flags, and 15,868 (5.2%) for ICD-10 codes. Definitions requiring dexamethasone, hypoxemia, or remdesivir selected sicker patients compared to PCR alone (mortality rates 12.2%, 10.7%, and 8.8% vs 8.3%, respectively). Definitions requiring PCR plus remdesivir or dexamethasone did not detect a reduction in in-hospital mortality associated with the SARS-CoV-2 Omicron variant. ICD-10 codes had the highest sensitivity (98.4%) but low specificity (39.5%) for distinguishing primary or contributing versus incidental COVID-19 hospitalizations. PCR plus dexamethasone had the highest specificity (92.1%) but low sensitivity (35.5%). Conclusions: Commonly used definitions for COVID-19 hospitalizations generate variable case counts and outcomes and differentiate poorly between primary or contributing versus incidental COVID-19 hospitalizations. Surveillance definitions that better capture and delineate COVID-19–associated hospitalizations are needed.

show abstract

“…Interestingly, phenotypes that relied on diagnosis code data performed less robustly. Previous studies have demonstrated the underreporting of conditions when relying on diagnostic codes alone [22][23][24]. Accordingly, it is possible that diagnostic codes themselves are not sensitive enough for identification of hypertension.…”

Section: Principal Findingsmentioning

confidence: 99%

Electronic Health Records for Population Health Management: Comparison of Electronic Health Record–Derived Hypertension Prevalence Measures Against Established Survey Data

Allen,

Valvi,

Gibson

et al. 2024

Online J Public Health Inform

View full text Add to dashboard Cite

Background Hypertension is the most prevalent risk factor for mortality globally. Uncontrolled hypertension is associated with excess morbidity and mortality, and nearly one-half of individuals with hypertension do not have the condition under control. Data from electronic health record (EHR) systems may be useful for community hypertension surveillance, filling a gap in local public health departments’ community health assessments and supporting the public health data modernization initiatives currently underway. To identify patients with hypertension, computable phenotypes are required. These phenotypes leverage available data elements—such as vitals measurements and medications—to identify patients diagnosed with hypertension. However, there are multiple methodologies for creating a phenotype, and the identification of which method most accurately reflects real-world prevalence rates is needed to support data modernization initiatives. Objective This study sought to assess the comparability of 6 different EHR-based hypertension prevalence estimates with estimates from a national survey. Each of the prevalence estimates was created using a different computable phenotype. The overarching goal is to identify which phenotypes most closely align with nationally accepted estimations. Methods Using the 6 different EHR-based computable phenotypes, we calculated hypertension prevalence estimates for Marion County, Indiana, for the period from 2014 to 2015. We extracted hypertension rates from the Behavioral Risk Factor Surveillance System (BRFSS) for the same period. We used the two 1-sided t test (TOST) to test equivalence between BRFSS- and EHR-based prevalence estimates. The TOST was performed at the overall level as well as stratified by age, gender, and race. Results Using both 80% and 90% CIs, the TOST analysis resulted in 2 computable phenotypes demonstrating rough equivalence to BRFSS estimates. Variation in performance was noted across phenotypes as well as demographics. TOST with 80% CIs demonstrated that the phenotypes had less variance compared to BRFSS estimates within subpopulations, particularly those related to racial categories. Overall, less variance occurred on phenotypes that included vitals measurements. Conclusions This study demonstrates that certain EHR-derived prevalence estimates may serve as rough substitutes for population-based survey estimates. These outcomes demonstrate the importance of critically assessing which data elements to include in EHR-based computer phenotypes. Using comprehensive data sources, containing complete clinical data as well as data representative of the population, are crucial to producing robust estimates of chronic disease. As public health departments look toward data modernization activities, the EHR may serve to assist in more timely, locally representative estimates for chronic disease prevalence.

show abstract

A multicenter evaluation of computable phenotyping approaches for SARS-CoV-2 infection and COVID-19 hospitalizations

Cited by 10 publications

References 35 publications

A computable case definition for patients with SARS-CoV2 testing that occurred outside the hospital

A computable case definition for patients with SARS-CoV2 testing that occurred outside the hospital

Impact of changing case definitions for coronavirus disease 2019 (COVID-19) hospitalization on pandemic metrics

Electronic Health Records for Population Health Management: Comparison of Electronic Health Record–Derived Hypertension Prevalence Measures Against Established Survey Data

Contact Info

Product

Resources

About