Transparent reporting of hypotheses and analyses in behavioral medicine research: an audit of publications in 2018 and 2008

Abstract

Objective: We aimed to document the use of transparent reporting of hypotheses and analyses in behavioral medicine journals in 2018 and 2008. Design: We examined a randomly selected portion of articles published in 2018 and 2008 by behavioral medicine journals with the highest impact factor, excluding manuscripts that were reviews or purely descriptive. Main Outcome Measures: We coded whether articles explicitly stated if the hypotheses/outcomes/analyses were primary or secondary; if study was registered/pre-registered; if 'exploratory' or a related term was used to describe analyses/aims; and if power analyses were reported. Results: We coded 162 manuscripts published in 2018 (87% observational and 12% experimental). Sixteen percent were explicit in describing hypotheses/outcomes/analyses as primary or secondary, 51% appeared to report secondary hypotheses/ outcomes/analyses but did not use term 'secondary,' and 33% were unclear. Registration occurred in 14% of studies, but 91% did not report which analyses were registered. 'Exploratory' or related term was used in 31% of studies. Power analyses were reported in 8% of studies. Compared to 2008 (n = 120), studies published in 2018 were more likely to be registered and less likely to be unclear if outcomes were primary or secondary. Conclusions: Behavioral medicine stakeholders should consider strategies to increase clarity of reporting, and particularly details that will inform readers if analyses were pre-planned or post-hoc. Study registration: https://osf.io/39ztn ARTICLE HISTORY

Open Science in Health Psychology and Behavioral Medicine: A Statement From the Behavioral Medicine Research Council

“…Open Science practices overall were low (Table 1), except for the relatively high number of articles published as gold open access in Annals of Behavioral Medicine and Health Psychology (48.3% and 51.1%, respectively). This result is consistent with an analysis of reporting practices in 2018 in these three journals plus the American Journal of Preventive Medicine, in which there was low occurrence of elements such as explicit description of analyses as primary or secondary (16% of 162 sampled papers) and if and when studies were registered (13.6%) (12).…”

“…However, study registration did increase from 2008 to 2018. [12] Annals of Behavioral Medicine, Health Psychology, and Translational Behavioral Medicine, are signees to the Transparency and Openness Promotion (TOP) Guidelines [13][14][15] (see Box 1), which establish guidelines for data citation; data, materials, and code transparency; design and analysis; preregistration; and replication. Psychosomatic Medicine will likely become a signatory in 2022 [16].…”

“…It is difficult to reconcile these observations as journals and funders have become more stringent in their reporting requirements and need for registration. However, study registration did increase from 2008 to 2018 (12). Annals of Behavioral Medicine , Health Psychology , and Translational Behavioral Medicine are signees to the TOP Guidelines (13–15) (see Box 1), which establish guidelines for data citation; data, materials, and code transparency; design and analysis; preregistration; and replication.…”

Open science in health psychology and behavioral medicine: A statement from the Behavioral Medicine Research Council.

“…Open Science practices overall were low (Table 1), except for the relatively high number of articles published as gold open access in Annals of Behavioral Medicine and Health Psychology (48.3% and 51.1%, respectively). This result is consistent with an analysis of reporting practices in 2018 in these three journals plus the American Journal of Preventive Medicine , in which there was low occurrence of elements such as explicit description of analyses as primary or secondary (16% of 162 sampled papers) and if and when studies were registered (13.6%) (McVay et al, 2021).…”

“…It is difficult to reconcile these observations as journals and funders have become more stringent in their reporting requirements and need for registration. However, study registration did increase from 2008 to 2018 (McVay et al, 2021). Annals of Behavioral Medicine , Health Psychology , and Translational Behavioral Medicine are signees to the TOP Guidelines (Nosek et al, 2015; Freedland, 2021; McVay & Conroy, 2021) (see Box 1), which establish guidelines for data citation; data, materials, and code transparency; design and analysis; preregistration; and replication.…”