Constructing a disease database and using natural language processing to capture and standardize free text clinical information

Abstract

The ability to extract critical information about an infectious disease in a timely manner is critical for population health research. The lack of procedures for mining large amounts of health data is a major impediment. The goal of this research is to use natural language processing (NLP) to extract key information (clinical factors, social determinants of health) from free text. The proposed framework describes database construction, NLP modules for locating clinical and non-clinical (social determinants) information, and a detailed evaluation protocol for evaluating results and demonstrating the effectiveness of the proposed framework. The use of COVID-19 case reports is demonstrated for data construction and pandemic surveillance. The proposed approach outperforms benchmark methods in F1-score by about 1–3%. A thorough examination reveals the disease’s presence as well as the frequency of symptoms in patients. The findings suggest that prior knowledge gained through transfer learning can be useful when researching infectious diseases with similar presentations in order to accurately predict patient outcomes.

Updated Surveillance Metrics and History of the COVID-19 Pandemic (2020-2023) in the Middle East and North Africa: Longitudinal Trend Analysis

“…To identify the appearance and duration of variants of concern, we used data on sequenced SARS-CoV-2 variants from the Global Initiative on Sharing All Influenza Data (GISAID), which is an effective and trusted online resource for sharing genetic, clinical, and epidemiological COVID-19 data [ 56 - 60 ]. We used Nextclade nomenclature [ 61 ] to collect clade designations from sequences and Pangolin nomenclature for lineage designations of SARS-CoV-2 [ 62 , 63 ].…”