ImportanceNonrandomized studies using insurance claims databases can be analyzed to produce real-world evidence on the effectiveness of medical products. Given the lack of baseline randomization and measurement issues, concerns exist about whether such studies produce unbiased treatment effect estimates.ObjectiveTo emulate the design of 30 completed and 2 ongoing randomized clinical trials (RCTs) of medications with database studies using observational analogues of the RCT design parameters (population, intervention, comparator, outcome, time [PICOT]) and to quantify agreement in RCT-database study pairs.Design, Setting, and ParticipantsNew-user cohort studies with propensity score matching using 3 US claims databases (Optum Clinformatics, MarketScan, and Medicare). Inclusion-exclusion criteria for each database study were prespecified to emulate the corresponding RCT. RCTs were explicitly selected based on feasibility, including power, key confounders, and end points more likely to be emulated with real-world data. All 32 protocols were registered on ClinicalTrials.gov before conducting analyses. Emulations were conducted from 2017 through 2022.ExposuresTherapies for multiple clinical conditions were included.Main Outcomes and MeasuresDatabase study emulations focused on the primary outcome of the corresponding RCT. Findings of database studies were compared with RCTs using predefined metrics, including Pearson correlation coefficients and binary metrics based on statistical significance agreement, estimate agreement, and standardized difference.ResultsIn these highly selected RCTs, the overall observed agreement between the RCT and the database emulation results was a Pearson correlation of 0.82 (95% CI, 0.64-0.91), with 75% meeting statistical significance, 66% estimate agreement, and 75% standardized difference agreement. In a post hoc analysis limited to 16 RCTs with closer emulation of trial design and measurements, concordance was higher (Pearson r, 0.93; 95% CI, 0.79-0.97; 94% meeting statistical significance, 88% estimate agreement, 88% standardized difference agreement). Weaker concordance occurred among 16 RCTs for which close emulation of certain design elements that define the research question (PICOT) with data from insurance claims was not possible (Pearson r, 0.53; 95% CI, 0.00-0.83; 56% meeting statistical significance, 50% estimate agreement, 69% standardized difference agreement).Conclusions and RelevanceReal-world evidence studies can reach similar conclusions as RCTs when design and measurements can be closely emulated, but this may be difficult to achieve. Concordance in results varied depending on the agreement metric. Emulation differences, chance, and residual confounding can contribute to divergence in results and are difficult to disentangle.

Studies that generate real-world evidence on the effects of medical products through analysis of digital data collected in clinical practice provide key insights for regulators, payers, and other healthcare decision-makers. Ensuring reproducibility of such findings is fundamental to effective evidence-based decision-making. We reproduce results for 150 studies published in peer-reviewed journals using the same healthcare databases as original investigators and evaluate the completeness of reporting for 250. Original and reproduction effect sizes were positively correlated (Pearson’s correlation = 0.85), a strong relationship with some room for improvement. The median and interquartile range for the relative magnitude of effect (e.g., hazard ratiooriginal/hazard ratioreproduction) is 1.0 [0.9, 1.1], range [0.3, 2.1]. While the majority of results are closely reproduced, a subset are not. The latter can be explained by incomplete reporting and updated data. Greater methodological transparency aligned with new guidance may further improve reproducibility and validity assessment, thus facilitating evidence-based decision-making. Study registration number: EUPAS19636.

Objectives A peer-support program called Resilience In Stressful Events (RISE) was designed to help hospital staff cope with stressful patient-related events. The aim of this study was to evaluate the impact of the RISE program by conducting an economic evaluation of its cost benefit. Methods A Markov model with a 1-year time horizon was developed to compare the cost benefit with and without the RISE program from a provider (hospital) perspective. Nursing staff who used the RISE program between 2015 and 2016 at a 1000-bed, private hospital in the United States were included in the analysis. The cost of running the RISE program, nurse turnover, and nurse time off were modeled. Data on costs were obtained from literature review and hospital data. Probabilities of quitting or taking time off with or without the RISE program were estimated using survey data. Net monetary benefit (NMB) and budget impact of having the RISE program were computed to determine cost benefit to the hospital. Results Expected model results of the RISE program found a net monetary benefit savings of US $22,576.05 per nurse who initiated a RISE call. These savings were determined to be 99.9% consistent on the basis of a probabilistic sensitivity analysis. The budget impact analysis revealed that a hospital could save US $1.81 million each year because of the RISE program. Conclusions The RISE program resulted in substantial cost savings to the hospital. Hospitals should be encouraged by these findings to implement institution-wide support programs for medical staff, based on a high demand for this type of service and the potential for cost savings.

No abstract

Background Medical and regulatory communities are increasingly interested in the utility of real-world evidence (RWE) for answering questions pertaining to drug safety and effectiveness but concerns about validity remain. A principled approach to conducting RWE studies may alleviate concerns and increase confidence in findings. This study sought to predict the findings from the PRONOUNCE trial using a principled approach to generating RWE. Methods This propensity-score (PS) matched observational cohort study utilized 3 claims databases to compare the occurrence of major adverse cardiovascular events (MACE) among initiators of degarelix vs. leuprolide. Patients were included if they had history of prostate cancer and atherosclerotic cardiovascular disease. Subjects were excluded if they didn’t have continuous database enrollment in the year prior to treatment initiation, were exposed to androgen deprivation therapy or experienced an acute cardiovascular event within 30 days prior to treatment initiation, or had a history or risk factors of QT prolongation. Results There were 12,448 leuprolide and 1,969 degarelix study-eligible patients before matching, with 1,887 in each arm after PS-matching. The results for MACE comparing degarelix to leuprolide in the observational analysis (hazard ratio= 1.35; 95% confidence interval = 0.94–1.93) was consistent with the subsequently released PRONOUNCE result (hazard ratio = 1.28; 95% confidence interval = 0.59–2.79). Conclusions This study successfully predicted the result of a comparative cardiovascular safety trial in the oncology setting. Although the findings are encouraging, limitations of measuring cancer stage and tumor progression are representative of challenges in attempting to generalize whether claims-based RWE can be used as actionable evidence.

No abstract

Background Many patients with cirrhosis have concurrent nonvalvular atrial fibrillation (NVAF). Data are lacking regarding recent oral anticoagulant (OAC) usage trends among US patients with cirrhosis and NVAF. Methods and Results Using MarketScan claims data (2012–2019), we identified patients with cirrhosis and NVAF eligible for OACs (CHA 2 DS 2 ‐VASc score ≥2 [men] or ≥3 [women]). We calculated the yearly proportion of patients prescribed a direct OAC (DOAC), warfarin, or no OAC. We stratified by high‐risk features (decompensated cirrhosis, thrombocytopenia, coagulopathy, chronic kidney disease, or end‐stage renal disease). Among 32 487 patients (mean age=71.6 years, 38.5% women, 15.1% with decompensated cirrhosis, mean CHA 2 DS 2 ‐VASc=4.2), 44.6% used OACs within 180 days of NVAF diagnosis, including DOACs (20.2%) or warfarin (24.4%). Compared with OAC nonusers, OAC users were less likely to have decompensated cirrhosis (18.6% versus 10.7%), thrombocytopenia (19.5% versus 12.5%), or chronic kidney disease/end‐stage renal disease (15.5% versus 14.0%). Between 2012 and 2019, warfarin use decreased by 21.0% (32.0% to 11.0%), whereas DOAC use increased by 30.6% (7.4% to 38.0%), and among all DOACs between 2012 and 2019, apixaban was the most commonly prescribed (46.1%). Warfarin use decreased and DOAC use increased in all subgroups, including in compensated and decompensated cirrhosis, thrombocytopenia, coagulopathy, chronic kidney disease/end‐stage renal disease, and across CHA 2 DS 2 ‐VASc categories. Among OAC users (2012–2019), DOAC use increased by 58.9% (18.7% to 77.6%). Among DOAC users, the greatest proportional increase was with apixaban (61.2%; P <0.001). Conclusions Among US patients with cirrhosis and NVAF, DOAC use has increased substantially and surpassed warfarin, including in decompensated cirrhosis. Nevertheless, >55% of patients remain untreated, underscoring the need for clearer treatment guidance.

Background In low-and-middle income countries (LMICs), accurate measures of the elements of quality care provided by a health worker through family planning services (also known as process quality) are required to ensure family’s contraceptives needs are being met. There are many tools used to assess family planning process quality of care (QoC) but no one standardized method. Those measuring QoC in LMICs should select an appropriate tool based the program context and financial/logistical parameters, but they require data on how well each tool measures routine clinical care. We aim to synthesize the literature on validity/comparability of family planning process QoC measurement tools through a quantitative systematic review with no meta-analysis. Methods We searched six literature databases for studies that compared quality measurements from different tools using quantitative statistics such as sensitivity/specificity, kappa statistic or absolute difference. We extracted the comparative measure along with other relevant study information, organized by quality indicator domain (e.g. counseling and privacy), and then classified the measure by low, medium, and high agreement. Results We screened 8172 articles and identified eight for analysis. Studies comparing quality measurements from simulated clients, direct observation, client exit interview, provider knowledge quizzes, and medical record review were included. These eight studies were heterogenous in their methods and the measurements compared. There was insufficient data to estimate overall summary measures of validity for the tools. Client exit interviews compared to direct observation or simulated client protocols had the most data and they were a poor proxy of the actual quality care received for many measurements. Conclusion To measure QoC consistently and accurately in LMICs, standardized tools and measures are needed along with an established method of combining them for a comprehensive picture of quality care. Data on how different tools proxy quality client care will inform these guidelines. Despite the small number of studies found during the review, we described important differences on how tools measure quality of care.