Outcome reporting bias in clinical trials: why monitoring matters

Ioannidis, John P. A.; Caplan, Arthur L.; Dal‐Ré, Rafael

doi:10.1136/bmj.j408

Cited by 61 publications

(45 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, if a researcher does not state anything about the handling of outliers, it may imply they will not remove any outlying values or that they will consider what to do with outliers after finding them, which means that the RDF is not restricted. Furthermore, studies on registrations of clinical trials show that other RDFs like outcome switching and HARKing are common in publications of preregistered studies (Chan, 2008;Chan & Altman, 2005;Chan, Hrobjartsson, Haahr, Gotzsche, & Altman, 2004;Chan, Hróbjartsson, Jørgensen, Gøtzsche, & Altman, 2008;Goldacre, 2016;Ioannidis, Caplan, & Dal-Ré, 2017;Jones et al, 2017;Lancee, Lemmens, Kahn, Vinkers, & Luykx, 2017;Mayo-Wilson et al, 2017;Rankin et al, 2017;Wayant et al, 2017). It is possible that increasing the explicit commitment to exhaustiveness will reduce this tendency.…”

Section: Discussionmentioning

confidence: 99%

Ensuring the quality and specificity of preregistrations

Bakker¹,

Veldkamp²,

Assen³

et al. 2018

Preprint

View full text Add to dashboard Cite

Researchers face many, often seemingly arbitrary choices in formulating hypotheses, designing protocols, collecting data, analyzing data, and reporting results. Opportunistic use of ‘researcher degrees of freedom’ aimed at obtaining statistical significance increases the likelihood of obtaining false positive results and overestimated effect sizes, and lowers the replicability of published results. Preregistration is a mechanism for reducing such degrees of freedom by specifying designs and analysis plans before observing the research outcomes. The effectiveness of preregistration may depend, in part, on whether the process facilitates sufficiently specific articulation of such plans. In this preregistered study, we compared two formats of preregistration available on the OSF: Standard Pre-Data Collection Registration and Prereg Challenge registration (now called “OSF Preregistration”, http://osf.io/prereg/). The Prereg Challenge format provides detailed instructions, a guided workflow, and an independent review to confirm completeness; the “Standard” format has minimal direct guidance to give researchers flexibility for what to pre-specify. Results of comparing random samples of 53 preregistrations from each format indicate that neither format restricted all researcher degrees of freedom and the Prereg Challenge format performed better on restricting degrees of freedom than the “Standard” format. We also found a very low concordance rate among coders about the number of hypotheses (about 15%), suggesting that preregistration is difficult and an acquired skill, and registration formats that provide effective guidance will improve the quality of research.

show abstract

Section: Discussionmentioning

confidence: 99%

Ensuring the quality and specificity of preregistrations

Bakker¹,

Veldkamp²,

Assen³

et al. 2018

Preprint

View full text Add to dashboard Cite

show abstract

“…The only way to ensure the absence (or minimization) of outcome reporting bias is by implementing better-quality control procedures during the editorial process, such as a thorough cross-checking between the manuscript and the protocol or registry [39]. Until this happens, clinicians should be prudent when interpreting the results of published trials and some systematic reviews/meta-analyses.…”

Section: Discussionmentioning

confidence: 99%

Why prudence is needed when interpreting articles reporting clinical trial results in mental health

Dal‐Ré

Bobes

Cuijpers

2017

Trials

Self Cite

View full text Add to dashboard Cite

BackgroundClinical trial results’ reliability is impacted by reporting bias. This is primarily manifested as publication bias and outcome reporting bias.Mental health trials’ specific featuresMental health trials are prone to two methodological deficiencies: (1) using small numbers of participants that facilitates false positive findings and exaggerated size effects, and (2) the obligatory use of psychometric scales that require subjective assessments. These two deficiencies contribute to the publication of unreliable results. Considerable reporting bias has been found in safety and efficacy findings in psychotherapy and pharmacotherapy trials. Reporting bias can be carried forward to meta-analyses, a key source for clinical practice guidelines. The final result is the frequent overestimation of treatment effects that could impact patients and clinician-informed decisions.Mechanisms to prevent outcome reporting biasProspective registration of trials and publication of results are the two major methods to reduce reporting bias. Prospective trial registration will allow checking whether they are published (so it will help to prevent publication bias) and, if published, whether those outcomes and analyses that were deemed as appropriate before trial commencement are actually published (hence helping to find out selective reporting of outcomes). Unfortunately, the rate of registered trials in mental health interventions is low and, frequently, of poor quality.ConclusionClinicians should be prudent when interpreting the results of published trials and some meta-analyses – such as those conducted by scientists working for the sponsor company or those that only include published trials. Prescribers, however, should be confident when prescribing drugs following the summary of product characteristics, since regulatory agencies have access to all clinical trial results.

show abstract

“…1 According to the authors, the various persons who can cross-check original protocol vs submitted manuscript information include editors, peer reviewers, external groups, and experts acknowledging that the task is so big that most of the journals do not have the kind of staff needed to do this. Therefore, simpler alternatives are needed and here we propose one.…”

Section: 5005/jp-journals-10028-1277mentioning

confidence: 99%

“…Addition/deletion/modification of prespecified outcomes makes clinical trial results untrustworthy, 1 but tackling this outcome reporting bias remains a challenge in spite of the advent of clinical trial registries. 2 A recent study, done nearly 20 years after the first registry was set up, highlighted the inconsistencies between registered and published outcomes and showed lack of correlation with funding agency as well as type of journals.…”

Section: Introudctionmentioning

confidence: 99%