Objectives To study the impact of blinding on estimated treatment effects, and their variation between trials; differentiating between blinding of patients, healthcare providers, and observers; detection bias and performance bias; and types of outcome (the MetaBLIND study). Design Meta-epidemiological study. Data source Cochrane Database of Systematic Reviews (2013-14). Eligibility criteria for selecting studies Meta-analyses with both blinded and non-blinded trials on any topic. Review methods Blinding status was retrieved from trial publications and authors, and results retrieved automatically from the Cochrane Database of Systematic Reviews. Bayesian hierarchical models estimated the average ratio of odds ratios (ROR), and estimated the increases in heterogeneity between trials, for non-blinded trials (or of unclear status) versus blinded trials. Secondary analyses adjusted for adequacy of concealment of allocation, attrition, and trial size, and explored the association between outcome subjectivity (high, moderate, low) and average bias. An ROR lower than 1 indicated exaggerated effect estimates in trials without blinding. Results The study included 142 meta-analyses (1153 trials). The ROR for lack of blinding of patients was 0.91 (95% credible interval 0.61 to 1.34) in 18 meta-analyses with patient reported outcomes, and 0.98 (0.69 to 1.39) in 14 meta-analyses with outcomes reported by blinded observers. The ROR for lack of blinding of healthcare providers was 1.01 (0.84 to 1.19) in 29 meta-analyses with healthcare provider decision outcomes (eg, readmissions), and 0.97 (0.64 to 1.45) in 13 meta-analyses with outcomes reported by blinded patients or observers. The ROR for lack of blinding of observers was 1.01 (0.86 to 1.18) in 46 meta-analyses with subjective observer reported outcomes, with no clear impact of degree of subjectivity. Information was insufficient to determine whether lack of blinding was associated with increased heterogeneity between trials. The ROR for trials not reported as double blind versus those that were double blind was 1.02 (0.90 to 1.13) in 74 meta-analyses. Conclusion No evidence was found for an average difference in estimated treatment effect between trials with and without blinded patients, healthcare providers, or outcome assessors. These results could reflect that blinding is less important than often believed or meta-epidemiological study limitations, such as residual confounding or imprecision. At this stage, replication of this study is suggested and blinding should remain a methodological safeguard in trials.

ObjectiveTo synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.DesignSystematic review of meta-epidemiological studies.MethodsWe retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome (“mortality” versus “other objective” versus “subjective”). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.ResultsWe included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.ConclusionsCertain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed.

http://www.controlled-trials.com , ISRCTN 23557269.

Randomized clinical trials underpin evidence‐based clinical practice, but flaws in their conduct may lead to biased estimates of intervention effects and hence invalid treatment recommendations. The main approach to the empirical study of bias is to collate a number of meta‐analyses and, within each, compare the results of trials with and without a methodological characteristic such as blinding of participants and health professionals. Estimated within‐meta‐analysis differences are combined across meta‐analyses, leading to an estimate of mean bias. Such “meta‐epidemiological” studies are published in increasing numbers and have the potential to inform trial design, assessment of risk of bias, and reporting guidelines. However, their interpretation is complicated by issues of confounding, imprecision, and applicability. We developed a guide for interpreting meta‐epidemiological studies, illustrated using MetaBLIND, a large study on the impact of blinding. Applying generally accepted principles of research methodology to meta‐epidemiology, we framed 10 questions covering the main issues to consider when interpreting results of such studies, including risk of systematic error, risk of random error, issues related to heterogeneity, and theoretical plausibility. We suggest that readers of a meta‐epidemiological study reflect comprehensively on the research question posed in the study, whether an experimental intervention was unequivocally identified for all included trials, the risk of misclassification of the trial characteristic, and the risk of confounding, i.e the adequacy of any adjustment for the likely confounders. We hope that our guide to interpretation of results of meta‐epidemiological studies is helpful for readers of such studies.

Highlights:• We show that Chinese women have higher AMH levels than the European counterparts before age of 25, but after the age of 25 this is reversed and Chinese women have lower AMH levels.• The disparity between the two populations widens with increasing age.Research Question: Chinese women are known to have an earlier age at natural menopause than their European counterparts, whether they also have a lower functional ovarian reserve is unknown. This study was designed to assess whether there are ethnic differences in Anti-Müllerian Hormone (AMH) in women of reproductive age.Design: Women with regular menstrual cycles, not on hormonal contraception or with any medical history of note, were recruited to provide a day 2-5 early follicular sample in China and Europe. AMH was determined using the Roche Elecsys assay.AMH decline was modelled with a linear, quadratic and quadratic with interaction on age equations to assess the impact of ethnicity.Results: 887 European and 461 Chinese women participated in the study. Despite the Chinese population being slightly younger 34.1±8.4 years than their European counterparts 34.8±8.9 years, their median AMH was lower 1.87 (IQR 0.28, 3.64) as compared to 2.11 (IQR 0.73, 3.96), with evidence of increasing discordance from age 25 years. In all regression models of the AMH age-related decline, there was evidence of a difference between Chinese and European women. Whilst AMH was 28.1% (95% CI; 18.2, 36.7%) lower in the Chinese population at age 30, this decline increased to 79.4% (95% CI; 75.4, 82.9%) at age 45.Conclusions: There were independent effects of age and ethnicity on serum AMH concentrations, with Chinese women having a substantially lower AMH in adult life than their European counterparts from age 25 onwards.

BackgroundWhen designing and analysing clinical trials, using previous relevant information, perhaps in the form of evidence syntheses, can reduce research waste. We conducted the INVEST (INVestigating the use of Evidence Synthesis in the design and analysis of clinical Trials) survey to summarise the current use of evidence synthesis in trial design and analysis, to capture opinions of trialists and methodologists on such use, and to understand any barriers.MethodsOur sampling frame was all delegates attending the International Clinical Trials Methodology Conference in November 2015. Respondents were asked to indicate (1) their views on the use of evidence synthesis in trial design and analysis, (2) their own use during the past 10 years and (3) the three greatest barriers to use in practice.ResultsOf approximately 638 attendees of the conference, 106 (17%) completed the survey, half of whom were statisticians. Support was generally high for using a description of previous evidence, a systematic review or a meta-analysis in trial design. Generally, respondents did not seem to be using evidence syntheses as often as they felt they should. For example, only 50% (42/84 relevant respondents) had used a meta-analysis to inform whether a trial is needed compared with 74% (62/84) indicating that this is desirable. Only 6% (5/81 relevant respondents) had used a value of information analysis to inform sample size calculations versus 22% (18/81) indicating support for this. Surprisingly large numbers of participants indicated support for, and previous use of, evidence syntheses in trial analysis. For example, 79% (79/100) of respondents indicated that external information about the treatment effect should be used to inform aspects of the analysis. The greatest perceived barrier to using evidence synthesis methods in trial design or analysis was time constraints, followed by a belief that the new trial was the first in the area.ConclusionsEvidence syntheses can be resource-intensive, but their use in informing the design, conduct and analysis of clinical trials is widely considered desirable. We advocate additional research, training and investment in resources dedicated to ways in which evidence syntheses can be undertaken more efficiently, offering the potential for cost savings in the long term.Electronic supplementary materialThe online version of this article (doi:10.1186/s13063-017-1955-y) contains supplementary material, which is available to authorized users.

Background: Longitudinal studies are crucial for identifying potential risk factors for infection with, and consequences of, COVID-19, but relationships can be biased if they are associated with invitation and response to data collection. We describe factors relating to questionnaire invitation and response in COVID-19 questionnaire data collection in a multigenerational birth cohort (the Avon Longitudinal Study of Parents and Children, ALSPAC). Methods: We analysed online questionnaires completed between the beginning of the pandemic and easing of the first UK lockdown by participants with valid email addresses who had not actively disengaged from the study. We assessed associations of pre-pandemic sociodemographic, behavioural, anthropometric and health-related factors with: i) being sent a questionnaire; ii) returning a questionnaire; and iii) item response (for specific questions). Analyses were conducted in three cohorts: the index children born in the early 1990s (now young adults; 41 variables assessed), their mothers (35 variables) and the mothers’ partners (27 variables). Results: Of 14,849 young adults, 41% were sent a questionnaire, of whom 57% returned one. Item response was >95%. In this cohort, 78% of factors were associated with being sent a questionnaire, 56% with returning one, and, as an example of item response, 20% with keyworker status response. For instance, children from mothers educated to degree-level had greater odds of being sent a questionnaire (OR=5.59; 95% CI=4.87-6.41), returning one (OR=1.60; 95% CI=1.31-1.95), and responding to items (e.g., keyworker status OR=1.65; 95% CI=0.88-3.04), relative to children from mothers with fewer qualifications. Invitation and response rates and associations were similar in all cohorts. Conclusions: These results highlight the importance of considering potential biases due to non-response when using longitudinal studies in COVID-19 research and interpreting results. We recommend researchers report response rates and factors associated with invitation and response in all COVID-19 observational research studies, which can inform sensitivity analyses.

BackgroundSurgical site infections (SSIs) are common, occurring in up to 25% of > 4 million operations performed in England each year. Previous trials of the effect of wound dressings on the risk of developing a SSI are of poor quality and underpowered.Methods/DesignThis study is a feasibility and pilot trial to examine the feasibility of a full trial that will compare simple dressings, no dressing and tissue-glue as a dressing. It is examining the overall acceptability of trial participation, identifying opportunities for refinement, testing the feasibility of and validating new outcome tools to assess SSI, wound management issues and patients’ wound symptom experiences. It is also exploring methods for avoiding performance bias and blinding outcome assessors by testing the feasibility of collecting wound photographs taken in theatre immediately after wound closure and, at 4–8 weeks after surgery, taken by participants themselves or their carers. Finally, it is identifying the main cost drivers for an economic evaluation of dressing types. Integrated qualitative research is exploring acceptability and reasons for non-adherence to allocation. Adults undergoing primary elective or unplanned abdominal general surgery or Caesarean section are eligible. The main exclusion criteria are abdominal or other major surgery less than three months before the index operation or contraindication to dressing allocation. The trial is scheduled to recruit for nine months. The findings will be used to inform the design of a main trial.DiscussionThis pilot trial is the first pragmatic study to randomise participants to no dressing or tissue-glue as a dressing versus a simple dressing. Early evidence from the ongoing pilot shows that recruitment is proceeding well and that the interventions are acceptable to participants. Combined with the qualitative findings, the findings will inform whether a main, large trial is feasible and, if so, how it should be designed.Trial registration ISRCTN49328913. Registered on 20 October 2015.Electronic supplementary materialThe online version of this article (doi:10.1186/s13063-017-2102-5) contains supplementary material, which is available to authorized users.