How to evaluate potential non-specific effects of vaccines: the quest for randomized trials or time for triangulation?

Abstract

Emerging evidence suggests that vaccines, in addition to their disease-specific effects, have important non-specific effects (NSEs), which contribute to their overall effect on mortality and morbidity. Immunological studies have shown that NSEs are biologically plausible. Many advocate that randomized controlled trials (RCTs) with overall mortality or morbidity as the outcome are the only way forward to confirm or refute NSEs. Areas covered: We discuss the limitations of using RCTs only as a tool to evaluate NSEs of vaccines. Such RCTs can be ethically problematic, they are time consuming and expensive. Furthermore, they only assess the NSEs in a given context, but it is inherent in the concept of NSEs that the NSEs of a given vaccine are modified by other immunomodulatory conditions. As an alternative, we propose that triangulation of RCTs and observational studies, merging multiple lines of evidence with different underlying bias structures, can build a strong argument for causality. We examine two examples related to measles vaccine and oral polio vaccine. Expert commentary: Using RCTs alone to evaluate NSEs of vaccines severely limits the possibilities for studying NSEs. Results from both RCTs and non-RCT studies should be triangulated to strengthen causal interpretation.

Developing the concept of beneficial non-specific effect of live vaccines with epidemiological studies

“…Hopefully, the interest will not disappear this time. There are huge potentials in studying systematically -in RCTs, in natural experiments, and in observational studies -the NSEs of different vaccines in different disease environment and with different interacting interventions with a view to further improve overall health and not just specific-disease immunity (31).…”

Tobacco-Free Duo Adult-Child Contract for Prevention of Tobacco Use Among Adolescents and Parents: Protocol for a Mixed-Design Evaluation

“…The protocol has several strengths, compared to similar studies. First, it aligns with the current increasing consensus on the importance of mixed methods [ 32 ] and of triangulation approaches in accruing robust evidence in the evaluation of interventions [ 33 - 35 ]. Two aspects of community trials call for triangulation approaches.…”

High-quality evidence to inform clinical practice

“…1 The published medical literature is filled with examples of associations between treatment and outcome identified in observational studies that were subsequently disproven by well conducted random ised controlled trials (RCTs). [2][3][4] Framed this way, findings from an analysis of cardiovascular guidelines, 5 which showed that less than 10% of American College of Cardiology and American Heart Association and less than 15% of European Society of Cardiology guideline recommendations were based on evidence from RCTs, should be concerning for anyone involved in clinical research or clinical care. Without evidence from RCTs, physicians are forced to base clinical decisions on flawed inference from clinical outcome data or simply rely on expert opinion.…”

“…But the experience of the past decade has shown that smaller efforts will not be adequate. studies have suggested different thresholds of an RR of 10 or higher, 2 or 5 or higher (or RR <0•2) 3 to avoid an RCT. We agree that larger effects are less likely to be explained by confounding factors, but regulatory agencies appear to have no explicit or implicit thresholds for approval without requiring further testing in RCTs.…”

“…We have shown how triangulation of multiple sources of evidence with different confounder structures can be used to show causality when RCTs are not feasible. 3 Ideally, health policy changes (eg, introducing or stopping vaccines) should be done in a manner that allows for proper assessment of their effect on overall health. For instance, concerns about the negative sideeffects of the human papillomavirus vaccine could have been addressed more appropriately if the first cohorts had been randomly allocated to Hertzel Gerstein and colleagues 1 propose that estimates of treatment effects with a relative risk (RR) of more than 4 might show that randomised controlled trials (RCTs) are not needed because confounders are less likely to obscure the true treatment effects when effect sizes are this large.…”