How do researchers determine the difference to be detected in superiority trials? Results of a survey from a panel of researchers

Abstract

BackgroundThere is currently no guidance for selecting a specific difference to be detected in a superiority trial. We explored 3 factors that in our opinion should influence the difference to be detected (type of outcome, patient age group, and presence of treatment side-effects), and 3 that should not (baseline level of risk, logistical difficulties, and cost of treatment).MethodsWe conducted an experimental survey using a factorial design among 380 corresponding authors of randomized controlled trials indexed in Medline. Two hypothetical vignettes were submitted to participants: one described a trial of a new analgesic in mild trauma injuries, the other described a trial of a new chemotherapy among cancer patients. The first vignette tested the baseline level of risk, patient age-group, patient recruitment difficulties, and treatment side-effects. The second tested the baseline level of risk, patient age-group, type of outcome, and cost of treatment. The respondents were asked to select the smallest gain of effectiveness that should be detected by the trial.ResultsIn vignette 1, respondents selected a median difference to be detected corresponding to an improvement of 7.0 % in pain control with the new treatment. In vignette 2, they selected a median difference to be detected corresponding to a reduction of 5.0 % in mortality or cancer recurrence with the new chemotherapy. In both vignettes, the difference to be detected decreased significantly with the baseline risk. The other factor influencing difference to be detected was the age group, but the impact of this factor was smaller. Cost, side-effects, outcome severity, or mention of logistical difficulties did not significantly impact the difference to be detected selected by participants.ConclusionsThree of the anticipated effects conformed to our expectations (the effect of patient age, and absence of effect of the cost of treatment and of patient recruitment difficulties) and the other three did not. These findings can guide future research in determining differences to be detected in trials that can translate to meaningful clinical decision-making.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0195-2) contains supplementary material, which is available to authorized users.

Optimal contrast analysis with heterogeneous variances and budget concerns

“…In addition to the two-sided test of difference for a contrast, it is of clinical importance to test the hypotheses for noninferiority and superiority between mean effects (Laster & Johnson [ 28 ], Mulla et al [ 29 ], Piaggio et al [ 30 ], Scott [ 31 ]). The problem of testing noninferiority and superiority can be unified by the following hypotheses when larger values of ψ are better: where ψ 0 is the non-inferiority or superiority threshold (Fleming et al [ 32 ], Gayet-Ageron et al [ 33 ], Gayet-Ageron et al [ 34 ], Gladstone & Vach [ 35 ], Wien [ 36 ]). When ψ 0 < 0, the rejection of the null hypothesis implies noninferiority against the reference margin.…”