For EHR-derived data to yield reliable real-world evidence, it needs to be of known and sufficiently high quality. Considering the impact of mortality data completeness on survival endpoints, we highlight the importance of data quality assessment and advocate benchmarking to the NDI.

Background Recently, van Walraven developed a weighted summary score (VW) based on the 30 comorbidities from the Elixhauser comorbidity system. One of the 30 comorbidities, cardiac arrhythmia, is currently excluded as a comorbidity indicator in administrative datasets such as the Nationwide Inpatient Sample (NIS), prompting us to examine the validity of the VW score and its use in the NIS. Methods Using data from the 2009 Maryland State Inpatient Database, we derived weighted summary scores to predict in-hospital mortality based on the full (30) and reduced (29) set of comorbidities and compared model performance of these and other comorbidity summaries in 2009 NIS data. Results Weights of our derived scores were not sensitive to the exclusion of cardiac arrhythmia. When applied to NIS data, models containing derived summary scores performed nearly identically (c statistics for 30 and 29 variable-derived summary scores: 0.804 and 0.802, respectively) to the model using all 29 comorbidity indicators (c = 0.809), and slightly better than the VW score (c = 0.793). Each of these models performed substantially better than those based on a simple count of Elixhauser comorbidities (c = 0.745) or a categorized count (0, 1, 2, or ≥3 comorbidities; c = 0.737). Conclusions The VW score and our derived scores are valid in the NIS and are statistically superior to summaries using simple co-morbidity counts. Researchers wishing to summarize the Elixhauser comorbidities with a single value should use the VW score or those derived in this study.

Infective third-stage larvae (L3i) of the human parasite Strongyloides stercoralis share many morphological, developmental, and behavioral attributes with Caenorhabditis elegans dauer larvae. The ‘dauer hypothesis’ predicts that the same molecular genetic mechanisms control both dauer larval development in C. elegans and L3i morphogenesis in S. stercoralis. In C. elegans, the phosphatidylinositol-3 (PI3) kinase catalytic subunit AGE-1 functions in the insulin/IGF-1 signaling (IIS) pathway to regulate formation of dauer larvae. Here we identify and characterize Ss-age-1, the S. stercoralis homolog of the gene encoding C. elegans AGE-1. Our analysis of the Ss-age-1 genomic region revealed three exons encoding a predicted protein of 1,209 amino acids, which clustered with C. elegans AGE-1 in phylogenetic analysis. We examined temporal patterns of expression in the S. stercoralis life cycle by reverse transcription quantitative PCR and observed low levels of Ss-age-1 transcripts in all stages. To compare anatomical patterns of expression between the two species, we used Ss-age-1 or Ce-age-1 promoter::enhanced green fluorescent protein reporter constructs expressed in transgenic animals for each species. We observed conservation of expression in amphidial neurons, which play a critical role in developmental regulation of both dauer larvae and L3i. Application of the PI3 kinase inhibitor LY294002 suppressed L3i in vitro activation in a dose-dependent fashion, with 100 µM resulting in a 90% decrease (odds ratio: 0.10, 95% confidence interval: 0.08–0.13) in the odds of resumption of feeding for treated L3i in comparison to the control. Together, these data support the hypothesis that Ss-age-1 regulates the development of S. stercoralis L3i via an IIS pathway in a manner similar to that observed in C. elegans dauer larvae. Understanding the mechanisms by which infective larvae are formed and activated may lead to novel control measures and treatments for strongyloidiasis and other soil-transmitted helminthiases.

PURPOSELarge, generalizable real-world data can enhance traditional clinical trial results. The current study evaluates reliability, clinical relevance, and large-scale feasibility for a previously documented method with which to characterize cancer progression outcomes in advanced non–small-cell lung cancer from electronic health record (EHR) data.METHODSPatients who were diagnosed with advanced non–small-cell lung cancer between January 1, 2011, and February 28, 2018, with two or more EHR-documented visits and one or more systemic therapy line initiated were identified in Flatiron Health’s longitudinal EHR-derived database. After institutional review board approval, we retrospectively characterized real-world progression (rwP) dates, with a random duplicate sample to ascertain interabstractor agreement. We calculated real-world progression-free survival, real-world time to progression, real-world time to next treatment, and overall survival (OS) using the Kaplan-Meier method (index date was the date of first-line therapy initiation), and correlations between OS and other end points were assessed at the patient level (Spearman’s ρ).RESULTSOf 30,276 eligible patients,16,606 (55%) had one or more rwP event. Of these patients, 11,366 (68%) had subsequent death, treatment discontinuation, or new treatment initiation. Correlation of real-world progression-free survival with OS was moderate to high (Spearman’s ρ, 0.76; 95% CI, 0.75 to 0.77; evaluable patients, n = 20,020), and for real-world time to progression correlation with OS was lower (Spearman’s ρ, 0.69; 95% CI, 0.68 to 0.70; evaluable patients, n = 11,902). Interabstractor agreement on rwP occurrence was 0.94 (duplicate sample, n = 1,065) and on rwP date 0.85 (95% CI, 0.81 to 0.89; evaluable patients n = 358 [patients with two independent event captures within 30 days]). Median rwP abstraction time from individual EHRs was 18.0 minutes (interquartile range, 9.7 to 34.4 minutes).CONCLUSIONWe demonstrated that rwP-based end points correlate with OS, and that rwP curation from a large, contemporary EHR data set can be reliable, clinically relevant, and feasible on a large scale.

IntroductionReal-world evidence derived from electronic health records (EHRs) is increasingly recognized as a supplement to evidence generated from traditional clinical trials. In oncology, tumor-based Response Evaluation Criteria in Solid Tumors (RECIST) endpoints are standard clinical trial metrics. The best approach for collecting similar endpoints from EHRs remains unknown. We evaluated the feasibility of a RECIST-based methodology to assess EHR-derived real-world progression (rwP) and explored non-RECIST-based approaches.MethodsIn this retrospective study, cohorts were randomly selected from Flatiron Health’s database of de-identified patient-level EHR data in advanced non-small cell lung cancer. A RECIST-based approach tested for feasibility (N = 26). Three non-RECIST approaches were tested for feasibility, reliability, and validity (N = 200): (1) radiology-anchored, (2) clinician-anchored, and (3) combined. Qualitative and quantitative methods were used.ResultsA RECIST-based approach was not feasible: cancer progression could be ascertained for 23% (6/26 patients). Radiology- and clinician-anchored approaches identified at least one rwP event for 87% (173/200 patients). rwP dates matched 90% of the time. In 72% of patients (124/173), the first clinician-anchored rwP event was accompanied by a downstream event (e.g., treatment change); the association was slightly lower for the radiology-anchored approach (67%; 121/180). Median overall survival (OS) was 17 months [95% confidence interval (CI) 14, 19]. Median real-world progression-free survival (rwPFS) was 5.5 months (95% CI 4.6, 6.3) and 4.9 months (95% CI 4.2, 5.6) for clinician-anchored and radiology-anchored approaches, respectively. Correlations between rwPFS and OS were similar across approaches (Spearman’s rho 0.65–0.66). Abstractors preferred the clinician-anchored approach as it provided more comprehensive context.ConclusionsRECIST cannot adequately assess cancer progression in EHR-derived data because of missing data and lack of clarity in radiology reports. We found a clinician-anchored approach supported by radiology report data to be the optimal, and most practical, method for characterizing tumor-based endpoints from EHR-sourced data.FundingFlatiron Health Inc., which is an independent subsidiary of the Roche group.Electronic supplementary materialThe online version of this article (10.1007/s12325-019-00970-1) contains supplementary material, which is available to authorized users.

Real-world evidence derived from electronic health records (EHRs) is increasingly recognized as a supplement to evidence generated from traditional clinical trials. In oncology, tumor-based Response Evaluation Criteria in Solid Tumors (RECIST) endpoints are collected in clinical trials. The best approach for collecting similar endpoints from EHRs remains unknown.We evaluated the feasibility of a traditional RECIST-based methodology to assess EHR-derived real-world progression (rwP) and explored non-RECIST-based approaches. In this retrospective study, cohorts were randomly selected from Flatiron Health's database of patient-level EHR data in advanced non-small cell lung cancer. A RECIST-based approach was tested for feasibility (N=26). Three non-RECIST abstraction approaches were tested for feasibility, reliability, and validity (N=200): (1) radiology-anchored, (2) clinician-anchored, and (3) combined.RECIST-based cancer progression could be ascertained from the EHRs of 23% of patients (6/26).In 87% of patients (173/200), at least one rwP event was identified using both the radiologyand clinician-anchored approaches. rwP dates matched 90% of the time. In 72% of patients (124/173), the first clinician-anchored rwP event was accompanied by a downstream event (e.g., treatment change); the association was slightly lower for the radiology-anchored approach (67%; 121/180). Median overall survival (OS) was 17 months (95% confidence interval [CI]: 14, 19). Median real-world progression-free survival (rwPFS) was 5.5 (95% CI: 4.6, 6.3) and 4.9 months (95% CI: 4.2, 5.6) for clinician-anchored and radiology-anchored approaches, respectively. Correlations between rwPFS and OS were similar across approaches (Spearman's rho: 0.65-0.66). Abstractors preferred the clinician-anchored approach as it provided more

The findings provide greater insight into the nonmotor features that contribute to the success of subthalamic nucleus DBS procedures from the patient's perspective and raise questions about the treatment focus and emphasis on symptom profiles in DBS candidacy evaluations.

Purpose The aim of this study was to assess the impact of missing death data on survival analyses conducted in an oncology EHR‐derived database. Methods The study was conducted using the Flatiron Health oncology database and the National Death Index (NDI) as a gold standard. Three analytic frameworks were evaluated in advanced non‐small cell lung cancer (aNSCLC) patients: median overall survival [mOS]), relative risk estimates conducted within the EHR‐derived database, and “external control arm” analyses comparing an experimental group augmented with mortality data from the gold standard to a control group from the EHR‐derived database only. The hazard ratios (HRs) obtained within the EHR‐derived database (91% sensitivity) and the external control arm analyses, were compared with results when both groups were augmented with mortality data from the gold standard. The above analyses were repeated using simulated lower mortality sensitivities to understand the impact of more extreme levels of missing deaths. Results Bias in mOS ranged from modest (0.6–0.9 mos.) in the EHR‐derived cohort with (91% sensitivity) to substantial when lower sensitivities were generated through simulation (3.3–9.7 mos.). Overall, small differences were observed in the HRs for the EHR‐derived cohort across comparative analyses when compared with HRs obtained using the gold standard data source. When only one treatment arm was subject to estimation bias, the bias was slightly more pronounced, but increased substantially when lower sensitivities were simulated. Conclusions The impact on survival analysis is minimal with high mortality sensitivity with only modest impact associated within external control arm applications.