Future of clinical development is on the verge of a major transformation due to convergence of large new digital data sources, computing power to identify clinically meaningful patterns in the data using efficient artificial intelligence and machine-learning algorithms, and regulators embracing this change through new collaborations. This perspective summarizes insights, recent developments, and recommendations for infusing actionable computational evidence into clinical development and health care from academy, biotechnology industry, nonprofit foundations, regulators, and technology corporations. Analysis and learning from publically available biomedical and clinical trial data sets, real-world evidence from sensors, and health records by machine-learning architectures are discussed. Strategies for modernizing the clinical development process by integration of AI- and ML-based digital methods and secure computing technologies through recently announced regulatory pathways at the United States Food and Drug Administration are outlined. We conclude by discussing applications and impact of digital algorithmic evidence to improve medical care for patients.

Biomarkers are physiologic, pathologic, or anatomic characteristics that are objectively measured and evaluated as an indicator of normal biologic processes, pathologic processes, or biological responses to therapeutic interventions. Recent advances in the development of mobile digitally connected technologies have led to the emergence of a new class of biomarkers measured across multiple layers of hardware and software. Quantified in ones and zeros, these “digital” biomarkers can support continuous measurements outside the physical confines of the clinical environment. The modular software–hardware combination of these products has created new opportunities for patient care and biomedical research, enabling remote monitoring and decentralized clinical trial designs. However, a systematic approach to assessing the quality and utility of digital biomarkers to ensure an appropriate balance between their safety and effectiveness is needed. This paper outlines key considerations for the development and evaluation of digital biomarkers, examining their role in clinical research and routine patient care.

BackgroundDespite the rapid adoption of immunotherapies in advanced non–small cell lung cancer (advNSCLC), knowledge gaps remain about their real‐world (rw) performance.MethodsThis retrospective, observational, multicenter analysis used the Flatiron Health deidentified electronic health record‐derived database of rw patients with advNSCLC who received treatment with PD‐1 and/or PD‐L1 (PD‐[L]1) inhibitors before July 1, 2017 (N = 5257) and had ≥6 months of follow‐up. The authors investigated PD‐(L)1 line of treatment and PD‐L1 testing rates and the relationship between overall survival (OS) and rw intermediate endpoints: progression‐free survival (rwPFS), rw time to progression (rwTTP), rw time to next treatment (rwTTNT), and rw time to discontinuation (rwTTD).ResultsFirst‐line PD‐(L)1 inhibitor use increased from 0% (in the third quarter of 2014 [Q3 2014]) to 42% (Q2 2017) over the study period. PD‐L1 testing also increased (from 3% in Q3 2015 to 70% in Q2 2017). The estimated median OS was 9.3 months (95% CI, 8.9‐9.8 months), and the estimated rwPFS was 3.2 months (95% CI, 3.1‐3.3 months). Longer OS and rwPFS were associated with ≥50% PD‐L1 percentage staining results. Correlations (⍴) between OS and intermediate endpoints were ⍴ = 0.75 (95% CI, 0.73‐0.76) for rwPFS and ⍴ = 0.60 (95% CI, 0.57‐0.63) for rwTTP, and, for treatment‐based intermediate endpoints, correlations were ⍴ = 0.60 (95% CI, 0.56‐0.64) for rwTTNT (N = 856) and ⍴ = 0.81 (95% CI, 0.80‐0.82) for rwTTD.ConclusionsThe use of first‐line PD‐(L)1 inhibitors and PD‐L1 testing has substantially increased, with better outcomes for patients who have ≥50% PD‐L1 percentage staining. Intermediate rw tumor‐dynamics estimates were moderately correlated with OS in patients with advNSCLC who received immunotherapy, highlighting the need for optimizing and standardizing rw endpoints to enhance the understanding of patient outcomes outside clinical trials.

No abstract

PURPOSE This pilot study examined the ability to operationalize the collection of real-world data to explore the potential use of real-world end points extracted from data from diverse health care data organizations and to assess how these relate to similar end points in clinical trials for immunotherapy-treated advanced non–small-cell lung cancer. PATIENTS AND METHODS Researchers from six organizations followed a common protocol using data from administrative claims and electronic health records to assess real-world end points, including overall survival (rwOS), time to next treatment, time to treatment discontinuation (rwTTD), time to progression, and progression-free survival, among patients with advanced non–small-cell lung cancer treated with programmed death 1/programmed death-ligand 1 inhibitors in real-world settings. Data sets included from 269 to 6,924 patients who were treated between January 2011 and October 2017. Results from contributors were anonymized. RESULTS Correlations between real-world intermediate end points (rwTTD and time to next treatment) and rwOS were moderate to high (range, 0.6 to 0.9). rwTTD was the most consistent end points as treatment detail was available in all data sets. rwOS at 1 year post–programmed death-ligand 1 initiation ranged from 40% to 57%. In addition, rwOS as assessed via electronic health records and claims data fell within the range of median OS values observed in relevant clinical trials. Data sources had been used extensively for research with ongoing data curation to assure accuracy and practical completeness before the initiation of this research. CONCLUSION These findings demonstrate that real-world end points are generally consistent with each other and with outcomes observed in randomized clinical trials, which substantiates the potential validity of real-world data to support regulatory and payer decision making. Differences observed likely reflect true differences between real-world and protocol-driven practices.

PURPOSELarge, generalizable real-world data can enhance traditional clinical trial results. The current study evaluates reliability, clinical relevance, and large-scale feasibility for a previously documented method with which to characterize cancer progression outcomes in advanced non–small-cell lung cancer from electronic health record (EHR) data.METHODSPatients who were diagnosed with advanced non–small-cell lung cancer between January 1, 2011, and February 28, 2018, with two or more EHR-documented visits and one or more systemic therapy line initiated were identified in Flatiron Health’s longitudinal EHR-derived database. After institutional review board approval, we retrospectively characterized real-world progression (rwP) dates, with a random duplicate sample to ascertain interabstractor agreement. We calculated real-world progression-free survival, real-world time to progression, real-world time to next treatment, and overall survival (OS) using the Kaplan-Meier method (index date was the date of first-line therapy initiation), and correlations between OS and other end points were assessed at the patient level (Spearman’s ρ).RESULTSOf 30,276 eligible patients,16,606 (55%) had one or more rwP event. Of these patients, 11,366 (68%) had subsequent death, treatment discontinuation, or new treatment initiation. Correlation of real-world progression-free survival with OS was moderate to high (Spearman’s ρ, 0.76; 95% CI, 0.75 to 0.77; evaluable patients, n = 20,020), and for real-world time to progression correlation with OS was lower (Spearman’s ρ, 0.69; 95% CI, 0.68 to 0.70; evaluable patients, n = 11,902). Interabstractor agreement on rwP occurrence was 0.94 (duplicate sample, n = 1,065) and on rwP date 0.85 (95% CI, 0.81 to 0.89; evaluable patients n = 358 [patients with two independent event captures within 30 days]). Median rwP abstraction time from individual EHRs was 18.0 minutes (interquartile range, 9.7 to 34.4 minutes).CONCLUSIONWe demonstrated that rwP-based end points correlate with OS, and that rwP curation from a large, contemporary EHR data set can be reliable, clinically relevant, and feasible on a large scale.

IntroductionReal-world evidence derived from electronic health records (EHRs) is increasingly recognized as a supplement to evidence generated from traditional clinical trials. In oncology, tumor-based Response Evaluation Criteria in Solid Tumors (RECIST) endpoints are standard clinical trial metrics. The best approach for collecting similar endpoints from EHRs remains unknown. We evaluated the feasibility of a RECIST-based methodology to assess EHR-derived real-world progression (rwP) and explored non-RECIST-based approaches.MethodsIn this retrospective study, cohorts were randomly selected from Flatiron Health’s database of de-identified patient-level EHR data in advanced non-small cell lung cancer. A RECIST-based approach tested for feasibility (N = 26). Three non-RECIST approaches were tested for feasibility, reliability, and validity (N = 200): (1) radiology-anchored, (2) clinician-anchored, and (3) combined. Qualitative and quantitative methods were used.ResultsA RECIST-based approach was not feasible: cancer progression could be ascertained for 23% (6/26 patients). Radiology- and clinician-anchored approaches identified at least one rwP event for 87% (173/200 patients). rwP dates matched 90% of the time. In 72% of patients (124/173), the first clinician-anchored rwP event was accompanied by a downstream event (e.g., treatment change); the association was slightly lower for the radiology-anchored approach (67%; 121/180). Median overall survival (OS) was 17 months [95% confidence interval (CI) 14, 19]. Median real-world progression-free survival (rwPFS) was 5.5 months (95% CI 4.6, 6.3) and 4.9 months (95% CI 4.2, 5.6) for clinician-anchored and radiology-anchored approaches, respectively. Correlations between rwPFS and OS were similar across approaches (Spearman’s rho 0.65–0.66). Abstractors preferred the clinician-anchored approach as it provided more comprehensive context.ConclusionsRECIST cannot adequately assess cancer progression in EHR-derived data because of missing data and lack of clarity in radiology reports. We found a clinician-anchored approach supported by radiology report data to be the optimal, and most practical, method for characterizing tumor-based endpoints from EHR-sourced data.FundingFlatiron Health Inc., which is an independent subsidiary of the Roche group.Electronic supplementary materialThe online version of this article (10.1007/s12325-019-00970-1) contains supplementary material, which is available to authorized users.

Background Evidence from cancer clinical trials has strong internal validity but can be difficult to generalize to real‐world patient populations. Here we analyzed real‐world outcomes of patients with metastatic non‐small cell lung cancer (mNSCLC) treated with programmed cell death protein 1 (PD‐1) inhibitors in the first year following U.S. regulatory approval. Materials and Methods This retrospective study leveraged electronic health record (EHR) data collected during routine patient care in community cancer care clinics. The cohort included patients with mNSCLC who had received nivolumab or pembrolizumab for metastatic disease (n = 1,344) with >1 EHR‐documented visit from January 1, 2011, to March 31, 2016. Patients with a > 90‐day gap between advanced disease diagnosis and first EHR structured data entry were excluded. Results Estimated median overall survival (OS) was 8.0 months (95% confidence interval 7.4–9.0 months). Estimated median OS was 4.7 months (3.4–6.6) for patients with anaplastic lymphoma kinase rearrangement‐ and epidermal growth factor receptor mutation‐positive tumors, and 8.6 months (7.7–10.6) for patients without such mutations. Age at PD‐1 inhibitor initiation or line of therapy did not impact OS. Conclusion This analysis suggests OS in real‐world patients may be shorter than in conventional clinical trial patient cohorts, potentially due to narrow trial eligibility criteria. The lack of difference in OS by line of therapy or age at immunotherapy initiation suggests sustained benefit of PD‐1 inhibitors in multitreated patients with mNSCLC and that age is not a predictor of outcome. Further studies are underway in patients with comorbidities, organ dysfunction, and multiple prior therapies. Implications for Practice This study evaluated data derived from electronic health records of patients with metastatic non‐small cell lung cancer treated with programmed cell death protein 1 (PD‐1) inhibitors in the year following regulatory approval. This real‐world cohort had shorter overall survival (OS) indexed to PD‐1 inhibitor initiation than reported in clinical trials. Late‐line treatment did not influence OS, and patients aged >75 at immunotherapy initiation did not have worse outcomes than younger patients. As new therapies enter clinical practice, real‐world data can complement clinical trial evidence providing information on generalizability and helping inform clinical treatment decisions.