BackgroundThe development of the capability and capacity to evaluate the outcomes of trials of complex interventions is a key priority of the National Institute for Health Research (NIHR) and the Medical Research Council (MRC). The evaluation of complex treatment programmes for mental illness (e.g. cognitive–behavioural therapy for depression or psychosis) not only is a vital component of this research in its own right but also provides a well-established model for the evaluation of complex interventions in other clinical areas. In the context of efficacy and mechanism evaluation (EME) there is a particular need for robust methods for making valid causal inference in explanatory analyses of the mechanisms of treatment-induced change in clinical outcomes in randomised clinical trials.ObjectivesThe key objective was to produce statistical methods to enable trial investigators to make valid causal inferences about the mechanisms of treatment-induced change in these clinical outcomes. The primary objective of this report is to disseminate this methodology, aiming specifically at trial practitioners.MethodsThe three components of the research were (1) the extension of instrumental variable (IV) methods to latent growth curve models and growth mixture models for repeated-measures data; (2) the development of designs and regression methods for parallel trials; and (3) the evaluation of the sensitivity/robustness of findings to the assumptions necessary for model identifiability. We illustrate our methods with applications from psychological and psychosocial intervention trials, keeping the technical details to a minimum, leaving the reporting of the more theoretical and mathematically demanding results for publication in appropriate specialist journals.ResultsWe show how to estimate treatment effects and introduce methods for EME. We explain the use of IV methods and principal stratification to evaluate the role of putative treatment effect mediators and therapeutic process measures. These results are extended to the analysis of longitudinal data structures. We consider the design of EME trials. We focus on designs to create convincing IVs, bearing in mind assumptions needed to attain model identifiability. A key area of application that has become apparent during this work is the potential role of treatment moderators (predictive markers) in the evaluation of treatment effect mechanisms for personalised therapies (stratified medicine). We consider the role of targeted therapies and multiarm trials and the use of parallel trials to help elucidate the evaluation of mediators working in parallel.ConclusionsIn order to demonstrate both efficacy and mechanism, it is necessary to (1) demonstrate a treatment effect on the primary (clinical) outcome, (2) demonstrate a treatment effect on the putative mediator (mechanism) and (3) demonstrate a causal effect from the mediator to the outcome. Appropriate regression models should be applied for (3) or alternative IV procedures, which account for unmeasured confounding, provided that a valid instrument can be identified. Stratified medicine may provide a setting where such instruments can be designed into the trial. This work could be extended by considering improved trial designs, sample size considerations and measurement properties.FundingThe project presents independent research funded under the MRC–NIHR Methodology Research Programme (grant reference G0900678).

Personalised (stratified) medicine and treatment-effect mechanisms evaluation are inextricably linked. Stratification without corresponding mechanisms evaluation lacks credibility. In the presence of mediator-outcome confounding, mechanisms evaluation is dependent on stratification for its validity. Both stratification and treatment-effect mediation can be evaluated using a biomarker stratified trial design together with detailed baseline measurement of all known prognostic biomarkers and other prognostic covariates. Direct and indirect (mediated) effects should be estimated through the use of IV methods (the IV being the predictive marker by treatment interaction) together with adjustments for all known prognostic markers (confounders) - the latter adjustments contributing to increased precision (as in a conventional analysis of treatment effects) rather than bias reduction.

This paper uses a randomized controlled trial to test whether doorstep canvassing can raise participation in kerbside recycling. Existing research shows that canvassing can confront negative attitudes, increase understanding and resolve structural obstacles, but there is less known about the longitudinal effects of such interventions, which may fall away over time. 194 streets in Trafford, in the North West of England, UK were randomly assigned into a treatment and a control group. All households in the treatment group were visited by canvassers who were trained to promote and encourage recycling. Recycling participation rates for all households were measured by observing bin set out rates over a three-week period. Measurement was done before and after the canvassing campaign and then again three months later to see if the intervention had been effective in raising participation rates. Random-effects multilevel regression models, controlling for baseline recycling, street size, deprivation and size of ethnic minority population, show that the canvassing raised recycling participation rates for the treatment group compared to the control group, but there was a decline in the impact of the intervention over time. The intervention was more effective on streets with low levels of recycling at baseline.

No abstract