Text-Based Identification of Herpes Zoster Ophthalmicus With Ocular Involvement in the Electronic Health Record: A Population-Based Study

Zheng, Chengyi; Sy, Lina S.; Tanenbaum, Hilary C.; Tian, Yun; Luo, Yi; Ackerson, Bradley; Tseng, Hung Fu

doi:10.1093/ofid/ofaa652

Cited by 2 publications

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Natural Language Processing vs Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Development and Validation in Real-World Data (Preprint)

Zheng,

Ackerson,

Qiu

et al. 2024

JMIR Medical Informatics

View full text Add to dashboard Cite

Background: Diagnosis codes and prescription data are used in algorithms to identify postherpetic neuralgia (PHN), a debilitating complication of herpes zoster (HZ). Because of the questionable accuracy of codes and prescription data, manual chart review is sometimes used to identify PHN in electronic health records (EHR), which can be costly and time-consuming.Objective: To develop and validate a natural language processing (NLP) algorithm for automatically identifying PHN from unstructured EHR data. To compare its performance with that of code-based methods.Methods: This retrospective study used EHR data from Kaiser Permanente Southern California, a large integrated healthcare system that serves over 4.8 million members. The source population included members aged ?50 years who received an incident HZ diagnosis and accompanying antiviral prescription between 2018-2020 and had ?1 encounter within 90-180 days of the incident HZ diagnosis. The study team manually reviewed the EHR and identified PHN cases. For NLP development and validation, 500 and 800 random samples from the source population were selected, respectively. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F-score, and Matthews correlation coefficient (MCC) of NLP and the code-based methods were evaluated using chart-reviewed results as the reference standard. Results:The NLP algorithm identified PHN cases with 90.9% sensitivity, 98.5% specificity, 82.0% PPV, and 99.3% NPV. The composite scores of the NLP algorithm were 0.89 (F-score) and 0.85 (MCC). The prevalences of PHN in the validation data were 6.9% (reference standard), 7.6% (NLP), and 5.4-13.1% (code-based). The code-based methods achieved 52.7-61.8% sensitivity, 89.8-98.4% specificity, 27.6-72.1% PPV, and 96.3-97.1% NPV. The F-scores and MCCs were ranged between 0.45-0.59 and 0.32-0.61, respectively. Conclusions:The automated NLP-based approach identified PHN cases from the EHR with good accuracy. This method could be useful in population-based PHN research.

show abstract

Natural Language Processing vs Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Development and Validation in Real-World Data (Preprint)

Zheng,

Ackerson,

Qiu

et al. 2024

JMIR Medical Informatics

View full text Add to dashboard Cite

show abstract

Natural Language Processing vs Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Development and Validation in Real-World Data (Preprint)

Zheng,

Ackerson,

Qiu

et al. 2024

Preprint

View full text Add to dashboard Cite

BACKGROUND Diagnosis codes and prescription data are used in algorithms to identify postherpetic neuralgia (PHN), a debilitating complication of herpes zoster (HZ). Because of the questionable accuracy of codes and prescription data, manual chart review is sometimes used to identify PHN in electronic health records (EHR), which can be costly and time-consuming. OBJECTIVE To develop and validate a natural language processing (NLP) algorithm for automatically identifying PHN from unstructured EHR data. To compare its performance with that of code-based methods. METHODS This retrospective study used EHR data from Kaiser Permanente Southern California, a large integrated healthcare system that serves over 4.8 million members. The source population included members aged ≥50 years who received an incident HZ diagnosis and accompanying antiviral prescription between 2018-2020 and had ≥1 encounter within 90-180 days of the incident HZ diagnosis. The study team manually reviewed the EHR and identified PHN cases. For NLP development and validation, 500 and 800 random samples from the source population were selected, respectively. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F-score, and Matthews correlation coefficient (MCC) of NLP and the code-based methods were evaluated using chart-reviewed results as the reference standard. RESULTS The NLP algorithm identified PHN cases with 90.9% sensitivity, 98.5% specificity, 82.0% PPV, and 99.3% NPV. The composite scores of the NLP algorithm were 0.89 (F-score) and 0.85 (MCC). The prevalences of PHN in the validation data were 6.9% (reference standard), 7.6% (NLP), and 5.4-13.1% (code-based). The code-based methods achieved 52.7-61.8% sensitivity, 89.8-98.4% specificity, 27.6-72.1% PPV, and 96.3-97.1% NPV. The F-scores and MCCs were ranged between 0.45-0.59 and 0.32-0.61, respectively. CONCLUSIONS The automated NLP-based approach identified PHN cases from the EHR with good accuracy. This method could be useful in population-based PHN research.

show abstract

Text-Based Identification of Herpes Zoster Ophthalmicus With Ocular Involvement in the Electronic Health Record: A Population-Based Study

Cited by 2 publications

References 28 publications

Natural Language Processing vs Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Development and Validation in Real-World Data (Preprint)

Natural Language Processing vs Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Development and Validation in Real-World Data (Preprint)

Natural Language Processing vs Diagnosis Code-Based Methods for Postherpetic Neuralgia Identification: Development and Validation in Real-World Data (Preprint)

Contact Info

Product

Resources

About