Evaluation of selective outcome reporting and trial registration practices among addiction clinical trials

Abstract

Background and Aims Selective outcome reporting occurs when trialists pre-specify primary and secondary outcomes during trial planning but alter the definitions in the published report. Here, we investigate selective outcome reporting in published addiction randomized controlled trials (RCTs) and evaluate whether particular funding sources are associated with an increased likelihood of selective outcome reporting. Design We conducted a cross-sectional study of published addiction clinical trials. A PubMed search was performed to identify RCTs in addiction journals from 2013 to 2017. Included studies used a randomized design to address one of the following topics: (1) drug, alcohol and tobacco addiction prevention, (2) stabilization following excessive use of a substance, (3) relapse prevention or (4) recovery maintenance.Setting Single-center, medical research institution. Participants Our sample included 162 RCTs that were prospectively registered with a clearly defined primary outcome. Measurement We extracted the following items from addiction RCTs: journal, funding source, trial registry number (if included), sample size, dates of subject enrollment, whether primary and secondary outcomes were denoted, all published outcomes, P-value for all outcomes and whether authors mentioned any deviations from the trial protocol as it related to RCT outcomes. Findings In total, 47 of 162 RCTs (29.0%) had at least one major discrepancy between the trial registry and published RCT. Overall, these 47 RCTs included 54 major discrepancies. The most common major discrepancy was demotion of a primary registered outcome (19/54, 35.2%). The majority of RCTs (132/162, 81.5%) were funded from public sources. Additionally, 166 RCTs were excluded from our sample because registration could not be confirmed. Conclusions There is evidence suggestive of selective outcome reporting in addiction randomized controlled trials (RCTs). The most common major discrepancies pertained to the primary outcome.

Preregistration Is Neither Sufficient nor Necessary for Good Science

“…For example, researchers may select which particular study or hypothesis to preregister; they may preregister a study only after running a long series of calibration tests that are not reported; they may preregister multiple studies or versions of the same study, then only report a selection; they may intentionally or unintentionally specify the hypotheses to be tested and analyses to be performed in loose terms, hence buying themselves flexibility in subsequent testing and analyses. Moreover, it is not uncommon for published research that was formally preregistered to deviate in material ways from the preregistration (e.g., using different exclusion criteria or a different statistical model than what was preregistered), without proper disclosure (Claesen et al, 2019; Ofosu & Posner, 2019; Vassar, Roberts, Cooper, Wayant, & Bibens, 2019). There is even a danger that some unscrupulous researcher would “preregister” a study or an analysis after it has been conducted or performed—a form of fraud known as “PARKing” (Preregistering After Results are Known; Yamada, 2018).…”

An Evaluation of Reporting Guidelines and Clinical Trial Registry Requirements Among Addiction Medicine Journals

“…2 Vassar et al 3 also found inadequate intervention reporting among randomized control trials (RCTs) investigating alcohol use disorder and evidence of selective outcome reporting bias among those examining addiction. 4 Likely as a result of reports such as those, the National Institute on Drug Abuse issued a statement in 2013 calling for improvement in reproducibility and transparency in research. 5 The importance of sound, quality evidence in addiction research cannot be overstated.…”

Commentary on Vassar et al. (2020): Cautionary observations on the pre‐registration revolution

“…The paper by Vassar et al . , along with findings from other academic disciplines , show that we are still a long way from resolving the credibility or reproducibility crisis that exists in many fields of empirical research. Specifically, these studies highlight the limited inroads made by pre‐registration, one of the main tools available to academic journals to improve the quality and integrity of published research.…”

“…However, if there is little or no oversight of what is actually reported in manuscripts from pre‐registered studies, and many continue to selectively report favorable outcomes (approximately a third of pre‐registered clinical trials, according to Vassar et al . ), then journals are aiding the authors of these manuscripts in deceiving readers into believing that the results presented have greater validity than those from unregistered studies. In this manner, journals that insist on pre‐registration but do little to ensure adherence might be doing more damage to research credibility than those that do not require registration at all.…”

“…If, as Vassar et al . found, many addiction journals do not require pre‐registration of studies and those that do implement it in a manner that still allows selective reporting of positive results, then researchers who adhere to their pre‐registered plans and produce null results will soon learn that their good intentions and additional efforts go unrewarded and might even be to the detriment of their careers.…”