Development and Validation of a High‐Quality Composite Real‐World Mortality Endpoint

Curtis, Melissa D.; Griffith, Sandra D.; Tucker, Melisa; Taylor, Michael D.; Capra, William B.; Carrigan, Gillis; Holzman, B. G.; Torres, Aracelis Z.; You, Paul; Arnieri, Brandon; Abernethy, Amy Pickar

doi:10.1111/1475-6773.12872

Cited by 173 publications

(200 citation statements)

References 7 publications

Supporting

Mentioning

190

Contrasting

Unclassified

Order By: Relevance

“…At the time of this study, the database included information from 250 cancer clinics, which consisted of approximately 775 unique sites of care in the United States, although academic centers were excluded from this analysis. The quality of mortality in the EHR‐derived database has been previously evaluated . The 2015 NDI data served as the gold standard data source.…”

Section: Methodsmentioning

confidence: 99%

“…In addition to the classification described above corresponding to the empirically observed sensitivity in the EHR‐derived data source (90.6% as compared with the gold standard), lower sensitivity datasets were generated through simulation (Figure , “Sampling Methods”). Bootstrap sampling (ie, sampling with replacement) was performed with 1000 iterations to simulate sensitivities of 63.4% (simulation 1) and 72.5% (simulation 2) by randomly selecting 30% and 20% of true positives (cell A), respectively.…”

Section: Methodsmentioning

confidence: 99%

“…In EHR‐derived databases, mortality data are collected in a structured format as part of routine clinical care. However, researchers often must augment incomplete EHR mortality data with other sources, such as national death indices and commercial sources . Ideally, the quality of the data source will have benchmarks against a gold standard.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An evaluation of the impact of missing deaths on overall survival analyses of advanced non–small cell lung cancer patients conducted in an electronic health records database

Carrigan¹,

Whipple²,

Taylor³

et al. 2019

Pharmacoepidemiology and Drug

Self Cite

View full text Add to dashboard Cite

Purpose The aim of this study was to assess the impact of missing death data on survival analyses conducted in an oncology EHR‐derived database. Methods The study was conducted using the Flatiron Health oncology database and the National Death Index (NDI) as a gold standard. Three analytic frameworks were evaluated in advanced non‐small cell lung cancer (aNSCLC) patients: median overall survival [mOS]), relative risk estimates conducted within the EHR‐derived database, and “external control arm” analyses comparing an experimental group augmented with mortality data from the gold standard to a control group from the EHR‐derived database only. The hazard ratios (HRs) obtained within the EHR‐derived database (91% sensitivity) and the external control arm analyses, were compared with results when both groups were augmented with mortality data from the gold standard. The above analyses were repeated using simulated lower mortality sensitivities to understand the impact of more extreme levels of missing deaths. Results Bias in mOS ranged from modest (0.6–0.9 mos.) in the EHR‐derived cohort with (91% sensitivity) to substantial when lower sensitivities were generated through simulation (3.3–9.7 mos.). Overall, small differences were observed in the HRs for the EHR‐derived cohort across comparative analyses when compared with HRs obtained using the gold standard data source. When only one treatment arm was subject to estimation bias, the bias was slightly more pronounced, but increased substantially when lower sensitivities were simulated. Conclusions The impact on survival analysis is minimal with high mortality sensitivity with only modest impact associated within external control arm applications.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An evaluation of the impact of missing deaths on overall survival analyses of advanced non–small cell lung cancer patients conducted in an electronic health records database

Carrigan¹,

Whipple²,

Taylor³

et al. 2019

Pharmacoepidemiology and Drug

Self Cite

View full text Add to dashboard Cite

show abstract

“…17,[38][39][40] These initial successes highlight the great potential of using RWE to complete databases and validate existing administrative data. 37,45 Linkage to the National Death Index permitted the validation of mortality data available in EHRs that were collected by Flatiron Health, and this created a more comprehensive resource. The most notable is the recent ASCO-led effort called the Cancer Learning Intelligence Network for Quality (CancerLinQ).…”

Section: Benefits Of Real-world Evidencementioning

confidence: 99%

“…Currently, there are widespread efforts led by both academic and industry leaders to improve oncologic data quality with RWE. 45 For sarcoma researchers and oncologists who choose to participate in this initiative, the Flatiron Health database can provide high-quality, real-world, longitudinal, and genomic data to advance sarcoma care and drug development. 40 CancerLinQ, a nonprofit subsidiary of ASCO, captures and aggregates data from EHRs via direct feeds and automatically enters the information into a series of cloud-based databases.…”

Section: Benefits Of Real-world Evidencementioning

confidence: 99%

The opportunities and shortcomings of using big data and national databases for sarcoma research

Lyu

Haider

Landman

et al. 2019

Cancer

View full text Add to dashboard Cite

The rarity and heterogeneity of sarcomas make performing appropriately powered studies challenging and magnify the significance of large databases in sarcoma research. Established large tumor registries and population-based databases have become increasingly relevant for answering clinical questions regarding sarcoma incidence, treatment patterns, and outcomes. However, the validity of large databases has been questioned and scrutinized because of the inaccuracy and wide variability of coding practices and the absence of clinically relevant variables. In addition, the utilization of large databases for the study of rare cancers such as sarcoma may be particularly challenging because of the known limitations of administrative data and poor overall data quality. Currently, there are several large national cancer databases, including the Surveillance, Epidemiology, and End Results database, the National Cancer Data Base of the American College of Surgeons and the American Cancer Society, and the National Program of Cancer Registries of the Centers for Disease Control and Prevention. These databases are often used for sarcoma research, but they are limited by their dependence on administrative or billing data, the lack of agreement between chart abstractors on diagnosis codes, and the use of preexisting documented hospital diagnosis codes for tumor registries, which lead to a significant underestimation of sarcomas in large data sets. Current and future initiatives to improve databases and big data applications for sarcoma research include increasing the utilization of sarcoma-specific registries and encouraging national initiatives to expand on real-world, evidence-based data sets. Cancer 2019;125:2926-2934.

show abstract

Association of Broad-Based Genomic Sequencing With Survival Among Patients With Advanced Non–Small Cell Lung Cancer in the Community Oncology Setting

Presley

Tang

Soulos

et al. 2018

JAMA

Self Cite

119

115

View full text Add to dashboard Cite

IMPORTANCE Broad-based genomic sequencing is being used more frequently for patients with advanced non-small cell lung cancer (NSCLC). However, little is known about the association between broad-based genomic sequencing and treatment selection or survival among patients with advanced NSCLC in a community oncology setting. OBJECTIVE To compare clinical outcomes between patients with advanced NSCLC who received broad-based genomic sequencing vs a control group of patients who received routine testing for EGFR mutations and/or ALK rearrangements alone. DESIGN, SETTING, AND PARTICIPANTS Retrospective cohort study of patients with chart-confirmed advanced NSCLC between January 1, 2011, and July 31, 2016, and who received care at 1 of 191 oncology practices across the United States using the Flatiron Health Database. Patients were diagnosed with stage IIIB/IV or unresectable nonsquamous NSCLC who received at least 1 line of antineoplastic treatment. EXPOSURES Receipt of either broad-based genomic sequencing or routine testing (EGFR and/or ALK only). Broad-based genomic sequencing included any multigene panel sequencing assay examining more than 30 genes prior to third-line treatment. MAIN OUTCOMES AND MEASURES Primary outcomes were 12-month mortality and overall survival from the start of first-line treatment. Secondary outcomes included frequency of genetic alterations and treatments received. RESULTS Among 5688 individuals with advanced NSCLC (median age, 67 years [interquartile range, 41-85], 63.6% white, 80% with a history of smoking); 875 (15.4%) received broad-based genomic sequencing and 4813 (84.6%) received routine testing. Among patients who received broad-based genomic sequencing, 4.5% received targeted treatment based on testing results, 9.8% received routine EGFR/ALK targeted treatment, and 85.1% received no targeted treatment. Unadjusted mortality rates at 12 months were 49.2% for patients undergoing broad-based genomic sequencing and 35.9% for patients undergoing routine testing. Using an instrumental variable analysis, there was no significant association between broad-based genomic sequencing and 12-month mortality (predicted probability of death at 12 months, 41.1% for broad-based genomic sequencing vs 44.4% for routine testing; difference −3.6% [95% CI, −18.4% to 11.1%]; P = .63). The results were consistent in the propensity score-matched survival analysis (42.0% vs 45.1%; hazard ratio, 0.92 [95% CI, 0.73 to 1.11]; P = .40) vs unmatched cohort (hazard ratio, 0.69 [95% CI, 0.62 to 0.77]; log-rank P < .001). CONCLUSIONS AND RELEVANCE Among patients with advanced non-small cell lung cancer receiving care in the community oncology setting, broad-based genomic sequencing directly informed treatment in a minority of patients and was not independently associated with better survival.

show abstract

Development and Validation of a High‐Quality Composite Real‐World Mortality Endpoint

Abstract: For EHR-derived data to yield reliable real-world evidence, it needs to be of known and sufficiently high quality. Considering the impact of mortality data completeness on survival endpoints, we highlight the importance of data quality assessment and advocate benchmarking to the NDI.

Cited by 173 publications

References 7 publications

An evaluation of the impact of missing deaths on overall survival analyses of advanced non–small cell lung cancer patients conducted in an electronic health records database

An evaluation of the impact of missing deaths on overall survival analyses of advanced non–small cell lung cancer patients conducted in an electronic health records database

The opportunities and shortcomings of using big data and national databases for sarcoma research

Association of Broad-Based Genomic Sequencing With Survival Among Patients With Advanced Non–Small Cell Lung Cancer in the Community Oncology Setting

Contact Info

Product

Resources

About