Natural language processing of clinical notes for identification of critical limb ischemia

Afzal, Naveed; Mallipeddi, Vishnu Priya; Sohn, Sunghwan; Liu, Hongfang; Chaudhry, Rajeev; Scott, Christopher G.; Kullo, Iftikhar J.; Arruda‐Olson, Adelaide M.

doi:10.1016/j.ijmedinf.2017.12.024

Cited by 86 publications

(52 citation statements)

References 38 publications

(48 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Past studies have demonstrated that NLP can be used to obtain valuable data for research that can be more accurate than ICD codes. [7,[24][25][26] This study supports these findings, identifying NLP as clearly superior for individual phenotype algorithms. As data volume and accuracy are critical for big data initiatives, it stands to reason that NLP-derived features will yield superior models for these endeavors.…”

Section: Discussionsupporting

confidence: 73%

Augmented Intelligence with Natural Language Processing Applied to Electronic Health Records is Useful for Identifying Patients with Non-Alcoholic Fatty Liver Disease at Risk for Disease Progression

Vleck

Chan

Coca

et al. 2019

Preprint

View full text Add to dashboard Cite

Objective: Electronic health record (EHR) systems contain structured data and unstructured documentation. Clinical insights can be derived from analyzing both but optimal methods for this have not been studied extensively. We compared various approaches to analyzing EHR data for non-alcoholic fatty liver disease (NAFLD). Materials and Methods:We compared analysis of structured and unstructured EHR data using natural language processing (NLP), free-text search, and diagnostic codes against expert adjudication as the reference standard.Results: Out of 38,575 patients, we identified 2,281 patients with NAFLD. From the remainder, 10,653 patients with similar data density were selected as a control group. NLP was more sensitive than ICD and text search (NLP 0.93 vs. ICD 0.28 vs. text search 0.81) with higher a F2 score (NLP 0.92 vs. ICD 0.34 vs. text search 0.81). 619 patients had suspected NAFLD documented in radiology notes not acknowledged in other forms of clinical documentation. Of these, 232 (37.5%) were found to have more advanced liver disease after a median of 1,057 days.Discussion: NLP-based approaches have superior accuracy in identifying NAFLD within the EHR compared to ICD/text search-based approaches. Suspected NAFLD on imaging is often not acknowledged in subsequent clinical documentation. Many such patients are later found to have more advanced liver disease.Conclusion: For identification of NAFLD, NLP performed better than alternative selection modalities and facilitated follow-on analysis of information flow. If accuracy can be proven to persist across clinical domains, NLP can identify patient phenotypes for biomedical research in an accurate and high-throughput manner.

show abstract

Section: Discussionsupporting

confidence: 73%

Augmented Intelligence with Natural Language Processing Applied to Electronic Health Records is Useful for Identifying Patients with Non-Alcoholic Fatty Liver Disease at Risk for Disease Progression

Vleck

Chan

Coca

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The results of this study indicate that existing clinical codes of DFU cannot be used to create a registry without supporting evidence, and alternative means should be sought. Natural language processing (NLP) of clinical notes was used to identify critical limb ischemia in the EMR of Mayo Clinic . NLP demonstrated a significantly higher PPV, and a similarly high sensitivity vs using ICD‐9 billing codes alone.…”

Section: Discussionmentioning

confidence: 99%

Validity of diagnostic codes and estimation of prevalence of diabetic foot ulcers using a large electronic medical record database

Cahn

Altaras

Agami

et al. 2018

Diabetes Metabolism Res

View full text Add to dashboard Cite

Aims To assess the validity of the diagnostic codes relating to diabetic foot ulcer (DFU) in the electronic medical records of a large integrated care provider and to assess the prevalence of DFU among its members. Materials and Methods Data were obtained from the diabetes registry of Maccabi Healthcare Services (MHS), a 2.1‐million‐member sick‐fund in Israel, which included 125 665 patients in 2015. We randomly selected and reviewed ~400 patient files from each of the following categories during study period: (1) had a diagnostic code of DFU; (2) had a diagnostic code, or clinical condition suggestive of DFU including: leg‐ulcer, amputation, DFU in quartiles proximate to 2015 or abnormality reported by nurse; (3) patients at high risk for DFU (age > 35 and one of the following: peripheral artery disease, neuropathy, DFU during 2011‐2014, eGFR<30 mL/min/m2 or foot deformity). The patients' charts were reviewed by study physicians, and DFU was validated or refuted. Results Relying upon diagnostic codes entered by physicians, the positive predictive value (PPV) was 73.1% (95% CI 67.6‐78.2), and the sensitivity was 48.2% (95% CI 45.8‐50.7%). The PPV of the diagnostic codes listed by podiatrists were significantly lower, while that of codes listed by nurses was higher but with lower sensitivity. The estimated annual prevalence of DFU in the diabetes registry of MHS was 1.2% (95%CI 1.0‐1.5%). Conclusions Diagnostic codes alone cannot be used reliably to create a DFU registry. Nevertheless, the data collected provide an estimate of the prevalence of DFU among patients included in the MHS diabetes registry.

show abstract

“…Because NLP pipelines and machine-learning approaches, in general, use expert-annotated text corpora with information coded by different experts, higher interrater reliability would increase the signal-to-noise ratio and, thus, improve semantic classification accuracy in natural free-text. In turn, NLP tools could be more effective in the identification of clinically relevant concepts hidden in clinical notes and corresponding biomedical literature and could be linked to computerized decision support systems for the implementation of evidence-based management strategies at the point of care [ 34 ].…”

Section: Discussionmentioning

confidence: 99%

Online Information Infrastructure Increases Inter-Rater Reliability of Medical Coders: A Quasi-Experimental Study (Preprint)

Varghese

Sandmann

Dugas

2017

Journal of Medical Internet Research

View full text Add to dashboard Cite

BackgroundMedical coding is essential for standardized communication and integration of clinical data. The Unified Medical Language System by the National Library of Medicine is the largest clinical terminology system for medical coders and Natural Language Processing tools. However, the abundance of ambiguous codes leads to low rates of uniform coding among different coders.ObjectiveThe objective of our study was to measure uniform coding among different medical experts in terms of interrater reliability and analyze the effect on interrater reliability using an expert- and Web-based code suggestion system.MethodsWe conducted a quasi-experimental study in which 6 medical experts coded 602 medical items from structured quality assurance forms or free-text eligibility criteria of 20 different clinical trials. The medical item content was selected on the basis of mortality-leading diseases according to World Health Organization data. The intervention comprised using a semiautomatic code suggestion tool that is linked to a European information infrastructure providing a large medical text corpus of >300,000 medical form items with expert-assigned semantic codes. Krippendorff alpha (Kalpha) with bootstrap analysis was used for the interrater reliability analysis, and coding times were measured before and after the intervention.ResultsThe intervention improved interrater reliability in structured quality assurance form items (from Kalpha=0.50, 95% CI 0.43-0.57 to Kalpha=0.62 95% CI 0.55-0.69) and free-text eligibility criteria (from Kalpha=0.19, 95% CI 0.14-0.24 to Kalpha=0.43, 95% CI 0.37-0.50) while preserving or slightly reducing the mean coding time per item for all 6 coders. Regardless of the intervention, precoordination and structured items were associated with significantly high interrater reliability, but the proportion of items that were precoordinated significantly increased after intervention (eligibility criteria: OR 4.92, 95% CI 2.78-8.72; quality assurance: OR 1.96, 95% CI 1.19-3.25).ConclusionsThe Web-based code suggestion mechanism improved interrater reliability toward moderate or even substantial intercoder agreement. Precoordination and the use of structured versus free-text data elements are key drivers of higher interrater reliability.

show abstract

Natural language processing of clinical notes for identification of critical limb ischemia

Cited by 86 publications

References 38 publications

Augmented Intelligence with Natural Language Processing Applied to Electronic Health Records is Useful for Identifying Patients with Non-Alcoholic Fatty Liver Disease at Risk for Disease Progression

Augmented Intelligence with Natural Language Processing Applied to Electronic Health Records is Useful for Identifying Patients with Non-Alcoholic Fatty Liver Disease at Risk for Disease Progression

Validity of diagnostic codes and estimation of prevalence of diabetic foot ulcers using a large electronic medical record database

Online Information Infrastructure Increases Inter-Rater Reliability of Medical Coders: A Quasi-Experimental Study (Preprint)

Contact Info

Product

Resources

About