Unpublished rating scales: A major source of bias in randomised controlled trials of treatments for schizophrenia

Marshall, Max; Lockwood, Austin; Bradley, Caroline; Adams, Clive E; Joy, CB; Fenton, Mark

doi:10.1192/bjp.176.3.249

Cited by 821 publications

(180 citation statements)

References 6 publications

(7 reference statements)

Supporting

Mentioning

166

Contrasting

Unclassified

Order By: Relevance

“…38 Further, the effect estimates decreased as the proportion of relevant studies contributing to the meta-analysis increased. Other research has investigated discrepancies due to unpublished versus published scales 39 and handling of baseline and end-point data. 40 …”

Section: Selective Outcome Reportingmentioning

confidence: 99%

Standard 2: Containing Risk of Bias

et al. 2012

View full text Add to dashboard Cite

DILEMMAThere is a crisis of credibility facing the child health research community because of the paucity of reliable estimates of the effects of interventions in children. Associations between risk of bias assessments and treatment effect estimates have important implications, for the clinician, and the families face important challenges as decisionmakers stemming from results that exaggerate treatment effectiveness or safety. Consequently, interventions that are not efficacious and potentially harmful may be prescribed, whereas interventions that truly are efficacious may be withheld. [1][2][3][4][5] Positive trends in pediatric research have been observed since the first trial was published in 1948. Specifically, there has been a substantial increase in the number of trials published over time, the proportion of randomized to nonrandomized controlled trials, and the proportion of child to adult trials. 6 Reporting of methods has also improved; however, methodological quality remains modest. 6 Three studies have specifically examined risk of bias in pediatric trials by using the Cochrane Risk of Bias tool. [7][8][9] The results are summarized in Table 1 by risk of bias domain. In 2 reviews, the overall risk of bias was unclear or high for the vast majority of trials. 7,8 Both of these articles also revealed that trials at high or unclear risk of bias had exaggerated treatment effects compared with those at low risk of bias. Sequence generation and allocation concealment appear to be the domains that are consistently problematic. Importantly, several variables have been found to be associated with risk of bias including source of funding (industry-sponsored research revealing higher risk of bias), nature of the interventions (behavioral/educational interventions having higher risk of bias), and number of authors (higher risk of bias with fewer authors). 9 These analyses demonstrate that there is substantial room for improvement in the methodological and reporting quality of pediatric trials. The association between risk of bias assessments and treatment effect estimates has important implications for decision-making both in terms of false-positive and false-negative results. In practice, this may result in unrealistic expectations of treatment benefit and safety. Inadequate reporting can also have an impact on systematic reviews, which are being used increasingly as a basis for informed decisionmaking. Although systematic reviews aim to be comprehensive and include all relevant studies, the number of studies contributing to various analyses, be they qualitative or quantitative, is often considerably smaller due to inadequate reporting.Ensuring methodological rigor and complete reporting is essential for informed clinical decision-making. It is critical that issues of bias are recognized and addressed by every person who conducts trials, reviews trials, funds trials, or uses trials to guide practice. This standards article was motivated by 2 key factors: (1) emerging evidence demonstrating methodological flaws a...

show abstract

Section: Selective Outcome Reportingmentioning

confidence: 99%

Standard 2: Containing Risk of Bias

et al. 2012

View full text Add to dashboard Cite

show abstract

“…The decision in favor of an instrument may have important consequences. Marshall et al [1] showed that in schizophrenia trials authors were more likely to report that treatment was superior to control when an unpublished instrument was used in the comparison, rather than a published instrument. Furthermore, the selection of instruments with good measurement properties will lead to the detection of smaller treatment effects, or more power to draw stronger conclusions, and therefore to better interpretation of study results.…”

Section: Introductionmentioning

confidence: 99%

“…It was our aim (1) to find all existing systematic reviews of measurement properties, (2) to appraise the quality of the review process of these reviews, (3) to describe if and how the authors of reviews assessed the methodological quality of the primary studies included in these reviews, (4) to describe if and how the authors of reviews evaluated the results of the primary studies, and (5) to describe if authors of reviews synthesized the above-mentioned data (steps 3 and 4) to come to an overall conclusion regarding the quality of each instrument.…”

Section: Introductionmentioning

confidence: 99%

Evaluation of the methodological quality of systematic reviews of health status measurement instruments

et al. 2009

View full text Add to dashboard Cite

A systematic review of measurement properties of health-status instruments is a tool for evaluating the quality of instruments. Our aim was to appraise the quality of the review process, to describe how authors assess the methodological quality of primary studies of measurement properties, and to describe how authors evaluate results of the studies. Literature searches were performed in three databases. One hundred and forty-eight reviews were included. The purpose of included reviews was to identify health status instruments used in an evaluative application and to report on the measurement properties of these instruments. Two independent reviewers selected the articles and extracted the data. Reviews were often of low quality: 22% of the reviews used one database, the search strategy was often poorly described, and in many cases it was not reported whether article selection (75%) and data extraction (71%) was done by two independent reviewers. In 11 reviews the methodological quality of the primary studies was evaluated for all measurement properties, and of these 11 reviews only 7 evaluated the results. Methods to evaluate the quality of the primary studies and the results differed widely. The poor quality of reviews hampers evidence-based selection of instruments. Guidelines for conducting and reporting systematic reviews of measurement properties should be developed.

show abstract

“…In social and psychological intervention studies, outcome measures are often subjective, variables may relate to latent constructs, and information may come from multiple sources (for example, participants or providers). While it is an issue in other areas of research, the influence on RCT results of the quality of subjective outcome measures in social and psychological intervention research has long been highlighted, given their prevalence in social and psychological intervention research [43]. Descriptions of the validity, reliability, and psychometric properties of such measures are therefore particularly useful for social and psychological intervention trials, especially when they are not widely available or discussed in the research literature [26,44].…”

Section: Aspects Of Internal Validitymentioning

confidence: 99%

Developing a Reporting Guideline for Social and Psychological Intervention Trials

et al. 2013

View full text Add to dashboard Cite

Social and psychological interventions are often complex. Understanding randomised controlled trials (RCTs) of these complex interventions requires a detailed description of the interventions tested and the methods used to evaluate them; however, RCT reports often omit, or inadequately report, this information. Incomplete and inaccurate reporting hinders the optimal use of research, wastes resources, and fails to meet ethical obligations to research participants and consumers. In this paper, we explain how reporting guidelines have improved the quality of reports in medicine, and describe the ongoing development of a new reporting guideline for RCTs: CONSORT-SPI (an Extension for social and psychological interventions). We invite readers to participate in the project by visiting our website, in order to help us reach the best-informed consensus on these guidelines (http://tinyurl.com/ CONSORT-study).

show abstract

Unpublished rating scales: A major source of bias in randomised controlled trials of treatments for schizophrenia

Abstract: Unpublished scales are a source of bias in schizophrenia trials.

Cited by 821 publications

References 6 publications

Standard 2: Containing Risk of Bias

Standard 2: Containing Risk of Bias

Evaluation of the methodological quality of systematic reviews of health status measurement instruments

Developing a Reporting Guideline for Social and Psychological Intervention Trials

Contact Info

Product

Resources

About