Clinical trials in COVID-19 management &amp; prevention: A meta-epidemiological study examining methodological quality

Honarmand, Kimia; Penn, J.B.; Agarwal, Arnav; Siemieniuk, Reed; Brignardello‐Petersen, Romina; Bartoszko, Jessica J; Zeraatkar, Dena; Agoritsas, Thomas; Burns, Karen E. A.; Fernando, Shannon M.; Foroutan, Farid; Ge, Long; Lamontagne, François; Jiménez-Mora, Mario A; Murthy, Srinivas; Yepes-Núñez, Juan José; Vandvik, Per Olav; Ye, Zhikang; Rochwerg, Bram

doi:10.1016/j.jclinepi.2021.07.002

Cited by 13 publications

(7 citation statements)

References 135 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior studies have examined the COVID-19 trial landscape, evaluating trial design quality [24,25], choice of outcome [26], and presenting descriptive statistics on COVID-19 trials characteristics [2,5,6,[8][9][10][11]. This is the first study to assess the prevalence of informative COVID-19 clinical trials.…”

Section: Discussionmentioning

confidence: 99%

How informative were early SARS-CoV-2 treatment and prevention trials? a longitudinal cohort analysis of trials registered on ClinicalTrials.gov

et al. 2022

View full text Add to dashboard Cite

Background Early in the SARS-CoV-2 pandemic, commentators warned that some COVID trials were inadequately conceived, designed and reported. Here, we retrospectively assess the prevalence of informative COVID trials launched in the first 6 months of the pandemic. Methods Based on prespecified eligibility criteria, we created a cohort of Phase 1/2, Phase 2, Phase 2/3 and Phase 3 SARS-CoV-2 treatment and prevention efficacy trials that were initiated from 2020-01-01 to 2020-06-30 using ClinicalTrials.gov registration records. We excluded trials evaluating behavioural interventions and natural products, which are not regulated by the U.S. Food and Drug Administration (FDA). We evaluated trials on 3 criteria of informativeness: potential redundancy (comparing trial phase, type, patient-participant characteristics, treatment regimen, comparator arms and primary outcome), trials design (according to the recommendations set-out in the May 2020 FDA guidance document on SARS-CoV-2 treatment and prevention trials) and feasibility of patient-participant recruitment (based on timeliness and success of recruitment). Results We included all 500 eligible trials in our cohort, 58% of which were Phase 2 and 84.8% were directed towards the treatment of SARS-CoV-2. Close to one third of trials met all three criteria and were deemed informative (29.9% (95% Confidence Interval 23.7–36.9)). The proportion of potentially redundant trials in our cohort was 4.1%. Over half of the trials in our cohort (56.2%) did not meet our criteria for high quality trial design. The proportion of trials with infeasible patient-participant recruitment was 22.6%. Conclusions Less than one third of COVID-19 trials registered on ClinicalTrials.gov during the first six months met all three criteria for informativeness. Shortcomings in trial design, recruitment feasibility and redundancy reflect longstanding weaknesses in the clinical research enterprise that were likely amplified by the exceptional circumstances of a pandemic.

show abstract

Section: Discussionmentioning

confidence: 99%

How informative were early SARS-CoV-2 treatment and prevention trials? a longitudinal cohort analysis of trials registered on ClinicalTrials.gov

et al. 2022

View full text Add to dashboard Cite

show abstract

“…We note that these attributes should not be interpreted as indicators of the rigor of the trials or the efficacy of the interventions being examined. Any appraisal of adherence to reporting guidelines (see Quinn et al, 2021) or evaluation of the strength of evidence (see Honarmand et al, 2021) for any of the individual studies in our sample or in the literature as a whole was beyond the scope of this study.…”

Section: Discussionmentioning

confidence: 99%

Open Science and COVID-19 Randomized Controlled Trials: Examining Open Access, Preprinting, and Data Sharing-Related Practices During the Pandemic

Borghi

Payne

Ren

et al. 2022

Preprint

View full text Add to dashboard Cite

The COVID-19 pandemic has brought substantial attention to the systems used to communicate biomedical research. In particular, the need to rapidly and credibly communicate research findings has led many stakeholders to encourage researchers to adopt open science practices such as posting preprints and sharing data. To examine the degree to which this has led to the adoption of such practices, we examined the openness of a sample of 539 published papers describing the results of randomized controlled trials testing interventions to prevent or treat COVID-19. The majority (56%) of the papers in this sample were free to read at the time of our investigation and 23.56% were preceded by preprints. However, there is no guarantee that the papers without an open license will be available without a subscription in the future, and only 49.61% of the preprints we identified were linked to the subsequent peer-reviewed version. Of the 331 papers in our sample with statements identifying if (and how) related datasets were available, only a paucity indicated that data was available in a repository that facilitates rapid verification and reuse. Our results demonstrate that, while progress has been made, there is still a significant mismatch between aspiration and the practice of open science in an important area of the COVID-19 literature.

show abstract

“…We describe the fragility index by subgroups of trial characteristics in Table 2. The median (IQR) fragility indexes of RCTs in treatment drugs was 2.5 (1-6); in others it was 4.5 (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18). In contrast, the median (IQR) fragility index of vaccine trials was 119 (61-139).…”

Section: The Fragility Index In Covid-19 Trialsmentioning

confidence: 99%

“…and S.S.) screened the titles and abstracts of all identified articles in accordance with the following prespecified eligibility criteria. The inclusion criteria were RCTs that (1) were superiority trials, (2) randomly assigned patients 1:1 into 2 parallel groups, (3) reported at least 1 dichotomous or time-toevent outcome as statistically significant in the abstract, and (4) tested an intervention for COVID-19.…”

Section: Study Selectionmentioning

confidence: 99%

“…Information about this newly discovered infectious disease has been widely reported in both traditional and social media, resulting in global awareness of a previously unknown respiratory infection and increased public perception of risk. This emergency situation has pressured researchers to conduct randomized clinical trials (RCTs) immediately, at various study scales and of varied quality . Regardless of the scale and quality of RCTs, the results of each received attention from the general public and health care researchers, via different media, and people alternated between optimism and despair based on the individual findings of these trials …”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Fragility of Statistically Significant Results in Randomized Clinical Trials for COVID-19

et al. 2022

View full text Add to dashboard Cite

IMPORTANCEInterpreting results from randomized clinical trials (RCTs) for COVID-19, which have been published rapidly and in vast numbers, is challenging during a pandemic. OBJECTIVETo evaluate the robustness of statistically significant findings from RCTs for COVID-19 using the fragility index. DESIGN, SETTING, AND PARTICIPANTSThis cross-sectional study included COVID-19 trial articles that randomly assigned patients 1:1 into 2 parallel groups and reported at least 1 binary outcome as significant in the abstract. A systematic search was conducted using PubMed to identify RCTs on COVID-19 published until August 7, 2021.EXPOSURES Trial characteristics, such as type of intervention (treatment drug, vaccine, or others), number of outcome events, and sample size. MAIN OUTCOMES AND MEASURES Fragility index. RESULTSOf the 47 RCTs for COVID-19 included, 36 (77%) were studies of the effects of treatment drugs, 5 (11%) were studies of vaccines, and 6 (13%) were of other interventions. A total of 138 235 participants were included in these trials. The median (IQR) fragility index of the included trials was 4(1-11). The medians (IQRs) of the fragility indexes of RCTs of treatment drugs, vaccines, and other interventions were 2.5 (1-6), 119 (61-139), and 4.5 (1-18), respectively. The fragility index among more than half of the studies was less than 1% of each sample size, although the fragility index as a proportion of events needing to change would be much higher. CONCLUSIONS AND RELEVANCEThis cross-sectional study found a relatively small number of events (a median of 4) would be required to change the results of COVID-19 RCTs from statistically significant to not significant. These findings suggest that health care professionals and policy makers should not rely heavily on individual results of RCTs for COVID-19.

show abstract

Clinical trials in COVID-19 management & prevention: A meta-epidemiological study examining methodological quality

Cited by 13 publications

References 135 publications

How informative were early SARS-CoV-2 treatment and prevention trials? a longitudinal cohort analysis of trials registered on ClinicalTrials.gov

How informative were early SARS-CoV-2 treatment and prevention trials? a longitudinal cohort analysis of trials registered on ClinicalTrials.gov

Open Science and COVID-19 Randomized Controlled Trials: Examining Open Access, Preprinting, and Data Sharing-Related Practices During the Pandemic

The Fragility of Statistically Significant Results in Randomized Clinical Trials for COVID-19

Contact Info

Product

Resources

About