Natural experimental studies are often recommended as a way of understanding the health impact of policies and other large scale interventions. Although they have certain advantages over planned experiments, and may be the only option when it is impossible to manipulate exposure to the intervention, natural experimental studies are more susceptible to bias. This paper introduces new guidance from the Medical Research Council to help researchers and users, funders and publishers of research evidence make the best use of natural experimental approaches to evaluating population health interventions. The guidance emphasises that natural experiments can provide convincing evidence of impact even when effects are small or take time to appear. However, a good understanding is needed of the process determining exposure to the intervention, and careful choice and combination of methods, testing of assumptions and transparent reporting is vital. More could be learnt from natural experiments in future as experience of promising but lesser used methods accumulates

There is a strong link between mental health and physical health, but little is known about the pathways from one to the other. We analyse the direct and indirect effects of past mental health on present physical health and past physical health on present mental health using lifestyle choices and social capital in a mediation framework. We use data on 10,693 individuals aged 50 years and over from six waves (2002-2012) of the English Longitudinal Study of Ageing. Mental health is measured by the Centre for Epidemiological Studies Depression Scale (CES) and physical health by the Activities of Daily Living (ADL). We find significant direct and indirect effects for both forms of health, with indirect effects explaining 10% of the effect of past mental health on physical health and 8% of the effect of past physical health on mental health. Physical activity is the largest contributor to the indirect effects. There are stronger indirect effects for males in mental health (9.9%) and for older age groups in mental health (13.6%) and in physical health (12.6%). Health policies aiming at changing physical and mental health need to consider not only the direct cross-effects but also the indirect cross-effects between mental health and physical health.

No abstract

There is some evidence to suggest that how a primary care physician is paid does affect his/her behaviour but the generalisability of these studies is unknown. Most policy changes in the area of payment systems are inadequately informed by research. Future changes to doctor payment systems need to be rigorously evaluated.

This paper examines the synthetic control method in contrast to commonly used difference‐in‐differences (DiD) estimation, in the context of a re‐evaluation of a pay‐for‐performance (P4P) initiative, the Advancing Quality scheme. The synthetic control method aims to estimate treatment effects by constructing a weighted combination of control units, which represents what the treated group would have experienced in the absence of receiving the treatment. While DiD estimation assumes that the effects of unobserved confounders are constant over time, the synthetic control method allows for these effects to change over time, by re‐weighting the control group so that it has similar pre‐intervention characteristics to the treated group.We extend the synthetic control approach to a setting of evaluation of a health policy where there are multiple treated units. We re‐analyse a recent study evaluating the effects of a hospital P4P scheme on risk‐adjusted hospital mortality. In contrast to the original DiD analysis, the synthetic control method reports that, for the incentivised conditions, the P4P scheme did not significantly reduce mortality and that there is a statistically significant increase in mortality for non‐incentivised conditions. This result was robust to alternative specifications of the synthetic control method. © 2015 The Authors. Health Economics published by John Wiley & Sons Ltd.

Objective To examine whether the introduction of payment by results (a fixed tariff case mix based payment system) was associated with changes in key outcome variables measuring volume, cost, and quality of care between 2003/4 and 2005/6.Setting Acute care hospitals in England.Design Difference-in-differences analysis (using a control group created from trusts in England and providers in Scotland not implementing payment by results in the relevant years); retrospective analysis of patient level secondary data with fixed effects models.Data sources English hospital episode statistics and Scottish morbidity records for 2002/3 to 2005/6.Main outcome measures Changes in length of stay and proportion of day case admissions as a proxy for unit cost; growth in number of spells to measure increases in output; and changes in in-hospital mortality, 30 day post-surgical mortality, and emergency readmission after treatment for hip fracture as measures of impact on quality of care.Results Length of stay fell more quickly and the proportion of day cases increased more quickly where payment by results was implemented, suggesting a reduction in the unit costs of care associated with payment by results. Some evidence of an association between the introduction of payment by results and growth in acute hospital activity was found. Little measurable change occurred in the quality of care indicators used in this study that can be attributed to the introduction of payment by results.Conclusion Reductions in unit costs may have been achieved without detrimental impact on the quality of care, at least in as far as these are measured by the proxy variables used in this study.

Difference-in-differences (DiD) estimators provide unbiased treatment effect estimates when, in the absence of treatment, the average outcomes for the treated and control groups would have followed parallel trends over time. This assumption is implausible in many settings. An alternative assumption is that the potential outcomes are independent of treatment status, conditional on past outcomes. This paper considers three methods that share this assumption: the synthetic control method, a lagged dependent variable (LDV) regression approach, and matching on past outcomes. Our motivating empirical study is an evaluation of a hospital pay-for-performance scheme in England, the best practice tariffs programme. The conclusions of the original DiD analysis are sensitive to the choice of approach. We conduct a Monte Carlo simulation study that investigates these methods' performance. While DiD produces unbiased estimates when the parallel trends assumption holds, the alternative approaches provide less biased estimates of treatment effects when it is violated. In these cases, the LDV approach produces the most efficient and least biased estimates.

The UK National Health Service introduced a pay for performance scheme for primary care providers in 2004/5. The scheme rewarded providers for the proportion of eligible patients who received appropriate treatment. Eligible patients were those who had been reported by the provider as having the relevant disease minus those they exception reported as not suitable for treatment. Using rich provider level data, we find that differences in reported disease rates between providers, and differences in exception rates both between and within providers, suggest gaming. Faced with ratio performance indicators, providers acted on denominators as well as numerators. Copyright � The Author(s). Journal compilation � Royal Economic Society 2010.