An Electronic Health Record Text Mining Tool to Collect Real‐World Drug Treatment Outcomes: A Validation Study in Patients With Metastatic Renal Cell Carcinoma

Abstract

Real‐world evidence can close the inferential gap between marketing authorization studies and clinical practice. However, the current standard for real‐world data extraction from electronic health records (EHRs) for treatment evaluation is manual review (MR), which is time‐consuming and laborious. Clinical Data Collector (CDC) is a novel natural language processing and text mining software tool for both structured and unstructured EHR data and only shows relevant EHR sections improving efficiency. We investigated CDC as a real‐world data (RWD) collection method, through application of CDC queries for patient inclusion and information extraction on a cohort of patients with metastatic renal cell carcinoma (RCC) receiving systemic drug treatment. Baseline patient characteristics, disease characteristics, and treatment outcomes were extracted and these were compared with MR for validation. One hundred patients receiving 175 treatments were included using CDC, which corresponded to 99% with MR. Calculated median overall survival was 21.7 months (95% confidence interval (CI) 18.7–24.8) vs. 21.7 months (95% CI 18.6–24.8) and progression‐free survival 8.9 months (95% CI 5.4–12.4) vs. 7.6 months (95% CI 5.7–9.4) for CDC vs. MR, respectively. Highest F1‐score was found for cancer‐related variables (88.1–100), followed by comorbidities (71.5–90.4) and adverse drug events (53.3–74.5), with most diverse scores on international metastatic RCC database criteria (51.4–100). Mean data collection time was 12 minutes (CDC) vs. 86 minutes (MR). In conclusion, CDC is a promising tool for retrieving RWD from EHRs because the correct patient population can be identified as well as relevant outcome data, such as overall survival and progression‐free survival.

Discontinuation of Antihypertensive Medications on the Outcome of Hospitalized Patients With Severe Acute Respiratory Syndrome-Coronavirus 2

“…Baseline characteristics, medication use, comorbidities, and outcomes were extracted from the Electronic Health Records, by a Natural language processing and text mining-based tool validated in previous study. 16 For details see methods in the Data Supplement .…”

Treosulfan-induced myalgia in pediatric hematopoietic stem cell transplantation identified by an electronic health record text mining tool

“…Also, it is possible that we underestimated the number of patients with myalgia and its severity, because the medical records were initially screened using the CDC. However, a recent study has shown that the use of this search engine is reliable and accurate 27 . Overestimation is highly unlikely, because all EHRs with a positive hit were screened and confirmed manually.…”

“…Furthermore, the use of an electronic health record text mining tool has proven to be helpful in tracking adverse events. More studies have been published using CTcue or a comparable tool to retrieve data from the EHR, including a validation study 27 , 30 , 31 . In the future, in case of a suspicion of the occurrence of a specific adverse event, a text mining tool can efficiently extract data from EHRs and can therefore quickly provide clarity on the relationship with the use of a particular drug.…”

Assessing the Risk of Repeat Victimization Using Structured and Unstructured Police Information

“…Research has demonstrated the incremental value of unstructured information over structured information in the prediction of a wide-range of topics. Combining unstructured and structured information improved the prediction of child abuse (Amrit et al, 2017), medication administration errors (Härkänen et al, 2020), suicide attempts (Adamou et al, 2018), use of health care services (Hatef et al, 2021), coronary artery disease (Jonnagaddala et al, 2015), and drug treatment outcomes (Laar et al, 2020). With regard to risk assessment of repeat victimization based on official police records, to the best of our knowledge, the incremental value of unstructured police information over structured police information is not yet investigated.…”