Background
In non-inferiority trials, there is a concern that intention-to-treat (ITT) analysis, by including participants who did not receive the planned interventions, may bias towards making the treatment and control arms look similar and lead to mistaken claims of non-inferiority. In contrast, per protocol (PP) analysis is viewed as less likely to make this mistake and therefore preferable in non-inferiority trials. In a systematic review of antibiotic non-inferiority trials, we compared ITT and PP analyses to determine which analysis was more conservative.
Methods
In a secondary analysis of a systematic review, we included non-inferiority trials that compared different antibiotic regimens, used absolute risk reduction (ARR) as the main outcome and reported both ITT and PP analyses. All estimates and confidence intervals (CIs) were oriented so that a negative ARR favored the control arm, and a positive ARR favored the treatment arm. We compared ITT to PP analyses results. The more conservative analysis between ITT and PP analyses was defined as the one having a more negative lower CI limit.
Results
The analysis included 164 comparisons from 154 studies. In terms of the ARR, ITT analysis yielded the more conservative point estimate and lower CI limit in 83 (50.6%) and 92 (56.1%) comparisons respectively. The lower CI limits in ITT analysis favored the control arm more than in PP analysis (median of − 7.5% vs. -6.9%, p = 0.0402). CIs were slightly wider in ITT analyses than in PP analyses (median of 13.3% vs. 12.4%, p < 0.0001). The median success rate was 89% (interquartile range IQR 82 to 93%) in the PP population and 44% (IQR 23 to 60%) in the patients who were included in the ITT population but excluded from the PP population (p < 0.0001).
Conclusions
Contrary to common belief, ITT analysis was more conservative than PP analysis in the majority of antibiotic non-inferiority trials. The lower treatment success rate in the ITT analysis led to a larger variance and wider CI, resulting in a more conservative lower CI limit. ITT analysis should be mandatory and considered as either the primary or co-primary analysis for non-inferiority trials.
Trial registration
PROSPERO registration number CRD42020165040.
Coronavirus disease 2019 (COVID-19) mortality and morbidity have been shown to increase with deprivation and impact non-White ethnicities more severely. Despite the extra risk Black, Asian and Minority Ethnicity (BAME) groups face in the pandemic, our current medical research system seems to prioritise innovation aimed at people of European descent. We found significant difficulties in assessing baseline demographics in clinical trials for COVID-19 vaccines, displaying a lack of transparency in reporting. Further, we found that most of these trials take place in high-income countries, with only 25 of 219 trials (11.4%) taking place in lower middle-or low-income countries. Trials for the current best vaccine candidates (BNT162b2, ChadOx1, mRNA-173) recruited 80.0% White participants. Underrepresentation of BAME groups in medical research will perpetuate historical distrust in healthcare processes, and poses a risk of unknown differences in efficacy and safety of these vaccines by phenotype. Limiting trial demographics and settings will mean a lack of global applicability of the results of COVID-19 vaccine trials, which will slow progress towards ending the pandemic.
Background
Antibiotic non-inferiority randomized controlled trials (RCTs) are used for approval of new antibiotics and making changes to antibiotic prescribing in clinical practice. We conducted a systematic review to assess the methodological and reporting quality of antibiotic non-inferiority RCTs.
Methods
We searched MEDLINE, Embase, the Cochrane Database of Systematic Reviews and the FDA drug database from inception until Nov 22, 2019 for non-inferiority RCTs comparing different systemic antibiotic therapies. Comparisons between antibiotic types, doses, administration routes or durations were included. Methodological and reporting quality indicators were based on the CONSORT reporting guidelines. Two independent reviewers extracted the data.
Results
The systematic review included 227 studies. Of these, 135 (59.5%) studies were supported by pharmaceutical industry. Only 83 (36.6%) studies provided a justification for the non-inferiority margin. Reporting of both intention-to-treat (ITT) and per-protocol (PP) analyses were done in 165 (72.7%) studies. The conclusion was misleading in 34 (15.0%) studies. The studies funded by pharmaceutical industry were less likely to be stopped early due to logistical reasons (3.0% vs. 19.1%, OR=0.13 95% CI 0.04-0.37) and to show inconclusive results (11.1% vs. 42.9%, OR=0.17 95% CI 0.08-0.33). The quality of studies decreased over time with respect to blinding, early stopping, reporting of ITT with PP analysis and having misleading conclusions.
Conclusions
There is room for improvement in the methodology and reporting of antibiotic non-inferiority trials. Quality can be improved across the entire spectrum from investigators, funding agencies, as well as during the peer-review process.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.