The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey

Randomized controlled trials comparing surgery to non-operative management in neurosurgery: a systematic review

“…This is similar to findings in cardiology, rheumatology, and gastroenterology [20]. Furthermore, a recent study of RCTs in spine surgery showed that statistical findings could be considered weak as the addition of only few events or non-events would have changed the significance of the reported finding [4]. …”

“…Most innovation in neurosurgery takes place without formalized oversight, which some justify given the unique nature of surgery, an idea referred to as “surgical exceptionalism” [15]. Perhaps as a result, RCTs in neurosurgery are conducted relatively infrequently, and their quality has been suggested to be poor [4, 12, 18]. This may be especially true for trials comparing neurosurgical procedures to non-operative management, rather than to a different neurosurgical procedure or the use of a medical device [7, 11, 22].…”

The fragility of statistically significant findings from randomized trials in head and neck surgery

“…In our study, the median FI score amongst the identified trials was 1, suggesting that a single chance occurrence could alter the statistical significance of the findings. Studies reviewing the FI score of RCTs in spine surgery (median FI 5 2, IQR 5 1-3), 10 sports medicine (median FI 5 2, IQR 5 1-2.8), 8 and critical care (median FI 5 1, IQR 5 1-3.5) 9 showed similar results. In comparison, the original descriptive study of the FI by Walsh et al, showed far superior results with a median FI score of 8 (IQR 5 0-109).…”

“…The developers of the FI have demonstrated that 24% of RCTs published in high‐impact journals ( New England Journal of Medicine, Lancet, Journal of the American Medical Association , and British Medical Journal ) hinge on three or fewer events, and that over 50% of trials had an FI score that was lower than the number of patients lost to follow‐up . Similar observations have also been made following systematic reviews of the critical care, cardiology, and orthopedics literature, although the FI tool has yet to be explored in the head and neck surgical patient population …”

“…5 Similar observations have also been made following systematic reviews of the critical care, cardiology, and orthopedics literature, although the FI tool has yet to be explored in the head and neck surgical patient population. [7][8][9][10] Our primary objective was to apply the FI and thus delineate the robustness of statistically significant RCT results in head and neck oncology. A secondary objective was to identify trial characteristics associated with both low and high FI score values.…”