An evaluation of the practice of transparency and reproducibility in addiction medicine literature

Adewumi, Mopileola Tomi; Vo, Nam; Tritz, Daniel; Beaman, Jason; Vassar, Matt

doi:10.1016/j.addbeh.2020.106560

Cited by 23 publications

(38 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Current rates of preregistration within the addictive behaviour literature are worryingly low; Adewumi et al . [84] report that just 3% of articles in addiction medicine were pre‐registered, and our own review of empirical research on alcohol‐related attentional bias revealed only one pre‐registered study. Despite the benefits, preregistration is not a panacea and requires careful oversight by authors, editors, and reviewers.…”

Section: A Case For Open Sciencementioning

confidence: 99%

Raising the bar: improving methodological rigour in cognitive alcohol research

et al. 2021

View full text Add to dashboard Cite

Background and Aims A range of experimental paradigms claim to measure the cognitive processes underpinning alcohol use, suggesting that heightened attentional bias, greater approach tendencies and reduced cue‐specific inhibitory control are important drivers of consumption. This paper identifies methodological shortcomings within this broad domain of research and exemplifies them in studies focused specifically on alcohol‐related attentional bias. Argument and analysis We highlight five main methodological issues: (i) the use of inappropriately matched control stimuli; (ii) opacity of stimulus selection and validation procedures; (iii) a credence in noisy measures; (iv) a reliance on unreliable tasks; and (v) variability in design and analysis. This is evidenced through a review of alcohol‐related attentional bias (64 empirical articles, 68 tasks), which reveals the following: only 53% of tasks use appropriately matched control stimuli; as few as 38% report their stimulus selection and 19% their validation procedures; less than 28% used indices capable of disambiguating attentional processes; 22% assess reliability; and under 2% of studies were pre‐registered. Conclusions Well‐matched and validated experimental stimuli, the development of reliable cognitive tasks and explicit assessment of their psychometric properties, and careful consideration of behavioural indices and their analysis will improve the methodological rigour of cognitive alcohol research. Open science principles can facilitate replication and reproducibility in alcohol research.

show abstract

Section: A Case For Open Sciencementioning

confidence: 99%

Raising the bar: improving methodological rigour in cognitive alcohol research

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Preregistration can be initiated for both confirmatory and exploratory research. Current rates of preregistration within the addictive behaviour literature are worryingly low; Adewumi et al [84] report that just 3% of articles in addiction medicine were pre-registered, and our own review of empirical research on alcohol-related attentional bias revealed only one pre-registered study. Despite the benefits, preregistration is not a panacea and requires careful oversight by authors, editors, and reviewers.…”

Section: A Case For Open Sciencementioning

confidence: 72%

Raising the Bar: Improving Methodological Rigour in Cognitive Alcohol Research

Pennington¹,

Jones²,

Bartlett³

et al. 2021

Preprint

View full text Add to dashboard Cite

Background & Aims: A range of experimental paradigms claim to measure the cognitive processes underpinning alcohol use, suggesting that heightened attentional bias, greater approach tendencies and reduced cue-specific inhibitory control are important drivers of consumption. This paper identifies substantial methodological shortcomings within this broad domain of research, however, and exemplifies them in studies focused specifically on alcohol-related attentional bias. Argument/Analysis: We highlight five main methodological issues: (i) the use of inappropriately matched control stimuli; (ii) opacity of stimulus selection and validation procedures; (iii) a credence in noisy measures; (iv) a reliance on unreliable tasks; and (v) variability in design and analysis. This is evidenced through a review of alcohol-related attentional bias (64 empirical articles, 68 tasks), which reveals the following: only 53% of tasks utilise appropriately matched control stimuli; as few as 38% report their stimulus selection and 19% their validation procedures; less than 28% used indices capable of disambiguating attentional processes; 22% assess reliability; and under 2% of studies were preregistered. Conclusions: Substantial improvements are required to improve the methodological rigour of cognitive alcohol research. To facilitate this, we provide a practical guide for future research: we advocate the use of well-matched and validated experimental stimuli, the development of reliable cognitive tasks and explicit assessment of their psychometric properties, and careful consideration of behavioural indices and their analysis. Further, we discuss open science principles that can facilitate replication and reproducibility in alcohol research, thereby enhancing trust in a field that has significant implications for public health and policy.

show abstract

“…Compared to preregistration or Registered Reports, material or data sharing often requires considerably more resources and may be prohibitively burdensome for investigators with limited institutional resources or research funding. Not surprisingly then, several studies of journals covering mental health research have reported that while availability statements are common, the sharing of materials or data via public repositories is rare (Adewumi, Vo, Tritz, Beaman, & Vassar, 2021;Nutu et al, 2019;Sherry et al, 2020;Wallach, Boyack, & Ioannidis, 2018). Given concerns about marginalizing authors with inadequate resources for material or data sharing, and given limited uptake so far of these open science practices in scientific publishing, we believe it is premature for the IJED to mandate material or data sharing in open access repositories.…”

Section: Sharing Research Materials or Datamentioning

confidence: 99%

Open science practices for eating disorders research

Burke

Frank

Hilbert

et al. 2021

Intl J Eating Disorders

View full text Add to dashboard Cite

This editorial seeks to encourage the increased application of three open science practices in eating disorders research: Preregistration, Registered Reports, and the sharing of materials, data, and code. For each of these practices, we introduce updated International Journal of Eating Disorders author and reviewer guidance. Updates include the introduction of open science badges; specific instructions about how to improve transparency; and the introduction of Registered Reports of systematic or metaanalytical reviews. The editorial also seeks to encourage the study of open science practices. Open science practices pose considerable time and other resource burdens.Therefore, research is needed to help determine the value of these added burdens and to identify efficient strategies for implementing open science practices.

show abstract

An evaluation of the practice of transparency and reproducibility in addiction medicine literature

Cited by 23 publications

References 20 publications

Raising the bar: improving methodological rigour in cognitive alcohol research

Raising the bar: improving methodological rigour in cognitive alcohol research

Raising the Bar: Improving Methodological Rigour in Cognitive Alcohol Research

Open science practices for eating disorders research

Contact Info

Product

Resources

About