Doctors, patients, and other decision makers need access to the best available clinical evidence, which can come from systematic reviews, experimental trials, and observational research. Despite methodological challenges, high-quality observational studies have an important role in comparative effectiveness research because they can address issues that are otherwise difficult or impossible to study. In addition, many clinical and policy decisions do not require the very high levels of certainty provided by large, rigorous randomized trials. This paper provides insights and a framework to guide good decision making that involves the full range of high-quality comparative effectiveness research techniques, including observational research.
To complement real-world evidence (RWE) guidelines, the 2019 Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real-world Evidence (SPACE) framework elucidated a process for designing valid and transparent real-world studies. As an extension to SPACE, here, we provide a structured framework for conducting feasibility assessments-a step-by-step guide to identify decision grade, fit-for-purpose data, which complements the United States Food and Drug Administration (FDA)'s framework for a RWE program. The process was informed by our collective experience conducting systematic feasibility assessments of existing data sources for pharmacoepidemiology studies to support regulatory decisions. Used with the SPACE framework, the Structured Process to Identify Fit-For-Purpose Data (SPIFD) provides a systematic process for conducting feasibility assessments to determine if a data source is fit for decision making, helping ensure justification and transparency throughout study development, from articulation of a specific and meaningful research question to identification of fit-for-purpose data and study design. BACKGROUNDAccess to extensive and diverse real-world data (RWD) sources has grown exponentially over the past decade. [1][2][3] Receptivity to using RWD in real-world evidence (RWE) to complement clinical trial evidence has simultaneously increased, 4-7 leading to more frequent inclusion of RWD studies in regulatory and payer submission packages, 8,9 but with mixed success. Whereas particular therapeutic areas, such as oncology and rare diseases, have historically utilized RWE, advances are being made to understand the optimal settings for producing RWE fit for decision making by regulators, payers, and health technology assessment agencies. 10 Standardssuch as guidance documents, step-by-step processes, and templates, developed to guide researchers on the design and conduct of RWD studies-support validity and transparency, and ultimately bolster confidence in RWE. These good practices cover the continuum 11 from articulating a clear research question 12 to transparency in study conduct and reporting of results, [13][14][15][16] and include consideration of the hypothetical target trial, 12,17 identifying confounders by constructing causal diagrams, 12,18,19 identifying a fit-for-purpose design, 12,20 protocol development, [21][22][23][24][25][26][27] and visualizing the study design. 20 A Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent RWE (SPACE) framework elucidated a step-by-step process for designing valid and transparent real-world studies and provides templates to capture decision making and justification at each step. 12 The structured template for planning and reporting on the implementation of RWE studies (STaRT-RWE) picks up where SPACE leaves off, providing detailed templates to capture the final design and implementation details (e.g., specific algorithms for each study variable). Taken...
Patterns of missing data are seldom well-characterized in observational research. This study examined the magnitude of, and factors associated with, missing data across multiple observational studies. Missingness was evaluated for demographic, clinical, and patient-reported outcome (PRO) data from a procedure registry (TOPS), a rare disease (cystic fibrosis) registry (Port-CF), and a comparative effectiveness registry (glaucoma, RiGOR). Generalized linear mixed effects models were fit to assess whether patient characteristics or follow-up methods predicted missingness. Data from 156,707 surgical procedures, 32,118 cystic fibrosis patients, and 2373 glaucoma patients were analyzed. Data were rarely missing for demographics, treatments, and outcomes. Missingness for clinical variables varied by registry and measure and depended on whether a variable was required. Within RiGOR, PRO forms were missing more often when collected by e-mail compared with office-based paper data collection. In Port-CF, missingness varied based on insurance status and sex. Strategic consideration of operational approaches affecting missing data should be performed prior to data collection and assessed periodically during study conduct.
Purpose We evaluated the reproducibility of a study characterizing newly‐diagnosed multiple myeloma (MM) patients within an electronic health records (EHR) database using different analytic tools. Methods We reproduced the findings of a descriptive cohort study using an iterative two‐phase approach. In Phase I, a common protocol and statistical analysis plan (SAP) were implemented by independent investigators using the Aetion Evidence Platform® (AEP), a rapid‐cycle analytics tool, and SAS statistical software as a gold standard for statistical analyses. Using the UK Clinical Practice Research Datalink (CPRD) dataset, the study included patients newly diagnosed with MM within primary care setting and assessed baseline demographics, conditions, drug exposure, and laboratory procedures. Phase II incorporated analysis revisions based on our initial comparison of the Phase I findings. Reproducibility of findings was evaluate by calculating the match rate and absolute difference in prevalence between the SAS and AEP study results. Results Phase I yielded slightly discrepant results, prompting amendments to SAP to add more clarity to operational decisions. After detailed specification of data and operational choices, exact concordance was achieved for the number of eligible patients (N = 2646), demographics, comorbidities (i.e., osteopenia, osteoporosis, cardiovascular disease [CVD], and hypertension), bone pain, skeletal‐related events, drug exposure, and laboratory investigations in the Phase II analyses. Conclusions In this reproducibility study, a rapid‐cycle analytics tool and traditional statistical software achieved near‐exact findings after detailed specification of data and operational choices. Transparency and communication of the study design, operational and analytical choices between independent investigators were critical to achieve this reproducibility.
Transparency is increasingly promoted to instill trust in nonrandomized studies using real-world data. Graphics and data visualizations support transparency by aiding communication and understanding, and can inform study design and analysis decisions. However, other than graphical representation of a study design and flow diagrams (e.g., a Consolidated Standards of Reporting Trials [CONSORT] like diagram), specific standards on how to maximize validity and transparency with visualization are needed. This paper provides guidance on how to use visualizations throughout the life cycle of a pharmacoepidemiology study-from initial study design to final report-to facilitate rationalized and transparent decision-making about study design and implementation, and clear communication of study findings. Our intent is to help researchers align their practices with current consensus statements on transparency.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
customersupport@researchsolutions.com
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.