For EHR-derived data to yield reliable real-world evidence, it needs to be of known and sufficiently high quality. Considering the impact of mortality data completeness on survival endpoints, we highlight the importance of data quality assessment and advocate benchmarking to the NDI.
Background Recently, van Walraven developed a weighted summary score (VW) based on the 30 comorbidities from the Elixhauser comorbidity system. One of the 30 comorbidities, cardiac arrhythmia, is currently excluded as a comorbidity indicator in administrative datasets such as the Nationwide Inpatient Sample (NIS), prompting us to examine the validity of the VW score and its use in the NIS. Methods Using data from the 2009 Maryland State Inpatient Database, we derived weighted summary scores to predict in-hospital mortality based on the full (30) and reduced (29) set of comorbidities and compared model performance of these and other comorbidity summaries in 2009 NIS data. Results Weights of our derived scores were not sensitive to the exclusion of cardiac arrhythmia. When applied to NIS data, models containing derived summary scores performed nearly identically (c statistics for 30 and 29 variable-derived summary scores: 0.804 and 0.802, respectively) to the model using all 29 comorbidity indicators (c = 0.809), and slightly better than the VW score (c = 0.793). Each of these models performed substantially better than those based on a simple count of Elixhauser comorbidities (c = 0.745) or a categorized count (0, 1, 2, or ≥3 comorbidities; c = 0.737). Conclusions The VW score and our derived scores are valid in the NIS and are statistically superior to summaries using simple co-morbidity counts. Researchers wishing to summarize the Elixhauser comorbidities with a single value should use the VW score or those derived in this study.
Infective third-stage larvae (L3i) of the human parasite Strongyloides stercoralis share many morphological, developmental, and behavioral attributes with Caenorhabditis elegans dauer larvae. The ‘dauer hypothesis’ predicts that the same molecular genetic mechanisms control both dauer larval development in C. elegans and L3i morphogenesis in S. stercoralis. In C. elegans, the phosphatidylinositol-3 (PI3) kinase catalytic subunit AGE-1 functions in the insulin/IGF-1 signaling (IIS) pathway to regulate formation of dauer larvae. Here we identify and characterize Ss-age-1, the S. stercoralis homolog of the gene encoding C. elegans AGE-1. Our analysis of the Ss-age-1 genomic region revealed three exons encoding a predicted protein of 1,209 amino acids, which clustered with C. elegans AGE-1 in phylogenetic analysis. We examined temporal patterns of expression in the S. stercoralis life cycle by reverse transcription quantitative PCR and observed low levels of Ss-age-1 transcripts in all stages. To compare anatomical patterns of expression between the two species, we used Ss-age-1 or Ce-age-1 promoter::enhanced green fluorescent protein reporter constructs expressed in transgenic animals for each species. We observed conservation of expression in amphidial neurons, which play a critical role in developmental regulation of both dauer larvae and L3i. Application of the PI3 kinase inhibitor LY294002 suppressed L3i in vitro activation in a dose-dependent fashion, with 100 µM resulting in a 90% decrease (odds ratio: 0.10, 95% confidence interval: 0.08–0.13) in the odds of resumption of feeding for treated L3i in comparison to the control. Together, these data support the hypothesis that Ss-age-1 regulates the development of S. stercoralis L3i via an IIS pathway in a manner similar to that observed in C. elegans dauer larvae. Understanding the mechanisms by which infective larvae are formed and activated may lead to novel control measures and treatments for strongyloidiasis and other soil-transmitted helminthiases.
PURPOSELarge, generalizable real-world data can enhance traditional clinical trial results. The current study evaluates reliability, clinical relevance, and large-scale feasibility for a previously documented method with which to characterize cancer progression outcomes in advanced non–small-cell lung cancer from electronic health record (EHR) data.METHODSPatients who were diagnosed with advanced non–small-cell lung cancer between January 1, 2011, and February 28, 2018, with two or more EHR-documented visits and one or more systemic therapy line initiated were identified in Flatiron Health’s longitudinal EHR-derived database. After institutional review board approval, we retrospectively characterized real-world progression (rwP) dates, with a random duplicate sample to ascertain interabstractor agreement. We calculated real-world progression-free survival, real-world time to progression, real-world time to next treatment, and overall survival (OS) using the Kaplan-Meier method (index date was the date of first-line therapy initiation), and correlations between OS and other end points were assessed at the patient level (Spearman’s ρ).RESULTSOf 30,276 eligible patients,16,606 (55%) had one or more rwP event. Of these patients, 11,366 (68%) had subsequent death, treatment discontinuation, or new treatment initiation. Correlation of real-world progression-free survival with OS was moderate to high (Spearman’s ρ, 0.76; 95% CI, 0.75 to 0.77; evaluable patients, n = 20,020), and for real-world time to progression correlation with OS was lower (Spearman’s ρ, 0.69; 95% CI, 0.68 to 0.70; evaluable patients, n = 11,902). Interabstractor agreement on rwP occurrence was 0.94 (duplicate sample, n = 1,065) and on rwP date 0.85 (95% CI, 0.81 to 0.89; evaluable patients n = 358 [patients with two independent event captures within 30 days]). Median rwP abstraction time from individual EHRs was 18.0 minutes (interquartile range, 9.7 to 34.4 minutes).CONCLUSIONWe demonstrated that rwP-based end points correlate with OS, and that rwP curation from a large, contemporary EHR data set can be reliable, clinically relevant, and feasible on a large scale.
IntroductionReal-world evidence derived from electronic health records (EHRs) is increasingly recognized as a supplement to evidence generated from traditional clinical trials. In oncology, tumor-based Response Evaluation Criteria in Solid Tumors (RECIST) endpoints are standard clinical trial metrics. The best approach for collecting similar endpoints from EHRs remains unknown. We evaluated the feasibility of a RECIST-based methodology to assess EHR-derived real-world progression (rwP) and explored non-RECIST-based approaches.MethodsIn this retrospective study, cohorts were randomly selected from Flatiron Health’s database of de-identified patient-level EHR data in advanced non-small cell lung cancer. A RECIST-based approach tested for feasibility (N = 26). Three non-RECIST approaches were tested for feasibility, reliability, and validity (N = 200): (1) radiology-anchored, (2) clinician-anchored, and (3) combined. Qualitative and quantitative methods were used.ResultsA RECIST-based approach was not feasible: cancer progression could be ascertained for 23% (6/26 patients). Radiology- and clinician-anchored approaches identified at least one rwP event for 87% (173/200 patients). rwP dates matched 90% of the time. In 72% of patients (124/173), the first clinician-anchored rwP event was accompanied by a downstream event (e.g., treatment change); the association was slightly lower for the radiology-anchored approach (67%; 121/180). Median overall survival (OS) was 17 months [95% confidence interval (CI) 14, 19]. Median real-world progression-free survival (rwPFS) was 5.5 months (95% CI 4.6, 6.3) and 4.9 months (95% CI 4.2, 5.6) for clinician-anchored and radiology-anchored approaches, respectively. Correlations between rwPFS and OS were similar across approaches (Spearman’s rho 0.65–0.66). Abstractors preferred the clinician-anchored approach as it provided more comprehensive context.ConclusionsRECIST cannot adequately assess cancer progression in EHR-derived data because of missing data and lack of clarity in radiology reports. We found a clinician-anchored approach supported by radiology report data to be the optimal, and most practical, method for characterizing tumor-based endpoints from EHR-sourced data.FundingFlatiron Health Inc., which is an independent subsidiary of the Roche group.Electronic supplementary materialThe online version of this article (10.1007/s12325-019-00970-1) contains supplementary material, which is available to authorized users.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.