Characteristics of Randomized Clinical Trials in Surgery From 2008 to 2020

Abstract

IMPORTANCE Randomized clinical trials (RCTs) provide the highest level of evidence to evaluate 2 or more surgical interventions. Surgical RCTs, however, face unique challenges in design and implementation. OBJECTIVE To evaluate the design, conduct, and reporting of contemporary surgical RCTs. EVIDENCE REVIEW A literature search performed in the 2 journals with the highest impact factor in general medicine as well as 6 key surgical specialties was conducted to identify RCTs published between 2008 and 2020. All RCTs describing a surgical intervention in both experimental and control arms were included. The quality of included data was assessed by establishing an a priori protocol containing all the details to extract. Trial characteristics, fragility index, risk of bias (Cochrane Risk of Bias 2 Tool), pragmatism (Pragmatic Explanatory Continuum Indicator Summary 2 [PRECIS-2]), and reporting bias were assessed. FINDINGS A total of 388 trials were identified. Of them, 242 (62.4%) were registered; discrepancies with the published protocol were identified in 81 (33.5%). Most trials used superiority design (329 [84.8%]), and intention-to-treat as primary analysis (221 [56.9%]) and were designed to detect a large treatment effect (50.0%; interquartile range [IQR], 24.7%-63.3%). Only 123 trials (31.7%) used major clinical events as the primary outcome. Most trials (303 [78.1%]) did not control for surgeon experience; only 17 trials (4.4%) assessed the quality of the intervention. The median sample size was 122 patients (IQR, 70-245 patients). The median follow-up was 24 months (IQR, 12.0-32.0 months). Most trials (211 [54.4%]) had some concern of bias and 91 (23.5%) had high risk of bias. The mean (SD) PRECIS-2 score was 3.52 (0.65) and increased significantly over the study period. Most trials (212 [54.6%]) reported a neutral result; reporting bias was identified in 109 of 211 (51.7%). The median fragility index was 3.0 (IQR, 1.0-6.0). Multiplicity was detected in 175 trials (45.1%), and only 35 (20.0%) adjusted for multiple comparisons. CONCLUSIONS AND RELEVANCEIn this systematic review, the size of contemporary surgical trials was small and the focus was on minor clinical events. Trial registration remained suboptimal and discrepancies with the published protocol and reporting bias were frequent. Few trials controlled for surgeon experience or assessed the quality of the intervention.

A scattered landscape: assessment of the evidence base for 71 patient decision aids developed in a hospital setting

“…This setting likely explains many of the encountered challenges regarding low-quality—often non-RCT evidence—and the need for indirect comparison across treatments. A recent review of RCTs in surgery by Robinson et al [ 33 ] concludes that more than half of the reviewed RCTs had some concern with bias and about 20% had high risk of bias—a finding we may confirm. Also, Robinson et al report that about 80% of RCTs did not control for surgeon experience.…”

Applying the PRECIS-2 tool for self-declared ‘pragmatic’ acupuncture trials: protocol for a systematic review

“…Two previous studies reported the risk of bias and reporting quality of included trials with the results of PRECIS-2 assessments. 18 19 However, since this study will evaluate the methodological features of trials in terms of pragmatism rather than reporting the clinical effect of interventions or quality of trials, assessing the risk of bias and reporting quality would be non-essential. (2) Search terms that could mean pragmatic intention are not included in this study.…”

“…The limitations of this study are as follows: (1) This study will not assess the risk of bias and reporting quality. Two previous studies reported the risk of bias and reporting quality of included trials with the results of PRECIS-2 assessments 18 19. However, since this study will evaluate the methodological features of trials in terms of pragmatism rather than reporting the clinical effect of interventions or quality of trials, assessing the risk of bias and reporting quality would be non-essential.…”

Recent Advances in the Diagnosis, Staging, Treatment, and Prognosis of Advanced Gastric Cancer: A Literature Review

“…While other trials 54.6% reported a neutral result; reporting bias was identified in 51.7%. Multiplicity was detected in 45.1%, and only 20.0% adjusted for multiple comparisons ( 76 ). In another trial, 117 individuals were randomized between control (surgery alone), intra-arterial chemotherapy and systemic chemotherapy.…”