Systematic reviews and meta-analyses are a key element of evidence-based healthcare, yet they remain in some ways mysterious. Why did the authors select certain studies and reject others? What did they do to pool results? How did a bunch of insignificant findings suddenly become significant? This paper, along with a book 1 that goes into more detail, demystifies these and other related intrigues. A review earns the adjective systematic if it is based on a clearly formulated question, identifies relevant studies, appraises their quality and summarizes the evidence by use of explicit methodology. It is the explicit and systematic approach that distinguishes systematic reviews from traditional reviews and commentaries. Whenever we use the term review in this paper it will mean a systematic review. Reviews should never be done in any other way. In this paper we provide a step-by-step explanationthere are just five steps-of the methods behind reviewing, and the quality elements inherent in each step (Box 1). For purposes of illustration we use a published review concerning the safety of public water fluoridation, but we must emphasize that our subject is review methodology, not fluoridation. EXAMPLE: SAFETY OF PUBLIC WATER FLUORIDATION You are a public health professional in a locality that has public water fluoridation. For many years, your colleagues and you have believed that it improves dental health. Recently there has been pressure from various interest groups to consider the safety of this public health intervention because they fear that it is causing cancer. Public health decisions have been based on professional judgment and practical feasibility without explicit consideration of the scientific evidence. (This was yesterday; today the evidence is available in a York review 2,3 , identifiable on MEDLINE through the freely accessible PubMed clinical queries interface [http://www.ncbi.nlm.nib.gov/entrez/query/ static/clinical.html], under 'systematic reviews'.

Systematic reviews and meta-analyses are a key element of evidence-based healthcare, yet they remain in some ways mysterious. Why did the authors select certain studies and reject others? What did they do to pool results? How did a bunch of insignificant findings suddenly become significant? This paper, along with a book 1 that goes into more detail, demystifies these and other related intrigues. A review earns the adjective systematic if it is based on a clearly formulated question, identifies relevant studies, appraises their quality and summarizes the evidence by use of explicit methodology. It is the explicit and systematic approach that distinguishes systematic reviews from traditional reviews and commentaries. Whenever we use the term review in this paper it will mean a systematic review. Reviews should never be done in any other way. In this paper we provide a step-by-step explanationthere are just five steps-of the methods behind reviewing, and the quality elements inherent in each step (Box 1). For purposes of illustration we use a published review concerning the safety of public water fluoridation, but we must emphasize that our subject is review methodology, not fluoridation. EXAMPLE: SAFETY OF PUBLIC WATER FLUORIDATION You are a public health professional in a locality that has public water fluoridation. For many years, your colleagues and you have believed that it improves dental health. Recently there has been pressure from various interest groups to consider the safety of this public health intervention because they fear that it is causing cancer. Public health decisions have been based on professional judgment and practical feasibility without explicit consideration of the scientific evidence. (This was yesterday; today the evidence is available in a York review 2,3 , identifiable on MEDLINE through the freely accessible PubMed clinical queries interface [http://www.ncbi.nlm.nib.gov/entrez/query/ static/clinical.html], under 'systematic reviews'.

Objective Open angle glaucoma is one of the most common causes of blindness in industrialised nations. Treatments to lower ocular pressure are widely used in glaucoma prevention and treatment, despite conflicting evidence. Design We performed meta-analyses to reassess the effectiveness of pressure lowering treatment to delay the development of glaucoma in ocular hypertension, as well as progression of manifest open angle glaucoma. Data sources Medline, Embase, and the Cochrane Library. Selection of studies Eligible studies were randomised controlled trials with a concurrent untreated control group and information on time to glaucomatous changes to visual field and optic disc. Trial reports were reviewed independently by two investigators in an unblinded standardised manner. Results Meta-analysis of trials in ocular hypertension showed a significant preventive effect of reducing intraocular pressure on progression to glaucoma (hazard ratio 0.56, 95% confidence interval 0.39 to 0.81, P = 0.01; number needed to treat 12). Pooled data of studies in manifest glaucoma showed a significant delay of visual field deterioration (0.65, 0.49 to 0.87, P = 0.003; NNT = 7), with subgroup analysis showing a larger effect in patients with raised pressure and a reduced effect in normal tension glaucoma (subgroup comparison: not significant). Conclusions Lowering intraocular pressure in patients with ocular hypertension or manifest glaucoma is beneficial in reducing the risk of visual field loss in the long term.

IMPORTANCE When analyzing results of randomized clinical trials, the treatment with the greatest specific effect compared with its placebo control is considered to be the most effective one. Although systematic variations of improvements in placebo control groups would have important implications for the interpretation of placebo-controlled trials, the knowledge base on the subject is weak. OBJECTIVE To investigate whether different types of placebo treatments are associated with different responses using the studies of migraine prophylaxis for this analysis. DESIGN, SETTING, AND PARTICIPANTS We searched relevant sources through February 2012 and contacted the authors to identify randomized clinical trials on the prophylaxis of migraine with an observation period of at least 8 weeks after randomization that compared an experimental treatment with a placebo control group. We calculated pooled random-effects estimates according to the type of placebo for the proportions of treatment response. We performed meta-regression analyses to identify sources of heterogeneity. In a network meta-analysis, direct and indirect comparisons within and across trials were combined. Additional analyses were performed for continuous outcomes. EXPOSURE Active migraine treatment and the placebo control conditions. MAIN OUTCOMES AND MEASURES Proportion of treatment responders, defined as having an attack frequency reduction of at least 50%. Other available outcomes in order of preference included a reduction of 50% or greater in migraine days, the number of headache days, or headache score or a significant improvement as assessed by the patients or their physicians. RESULTS Of the 102 eligible trials, 23 could not be included in the meta-analyses owing to insufficient data. Sham acupuncture (proportion of responders, 0.38 [95% CI, 0.30-0.47]) and sham surgery (0.58 [0.37-0.77]) were associated with a more pronounced reduction of migraine frequency than oral pharmacological placebos (0.22 [0.17-0.28]) and were the only significant predictors of response in placebo groups in multivariable analyses (P = .005 and P = .001, respectively). Network meta-analysis confirmed that more patients reported response in sham acupuncture groups than in oral pharmacological placebo groups (odds ratio, 1.88 [95% CI, 1.30-2.72]). Corresponding analyses for continuous outcomes showed similar findings. CONCLUSIONS AND RELEVANCE Sham acupuncture and sham surgery are associated with higher responder ratios than oral pharmacological placebos. Clinicians who treat patients with migraine should be aware that a relevant part of the overall effect they observe in practice might be due to nonspecific effects and that the size of such effects might differ between treatment modalities.

No abstract